CN112001948B - Target tracking processing method and device - Google Patents
Target tracking processing method and device Download PDFInfo
- Publication number
- CN112001948B CN112001948B CN202010753234.9A CN202010753234A CN112001948B CN 112001948 B CN112001948 B CN 112001948B CN 202010753234 A CN202010753234 A CN 202010753234A CN 112001948 B CN112001948 B CN 112001948B
- Authority
- CN
- China
- Prior art keywords
- target
- video frame
- motion state
- next video
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 230000033001 locomotion Effects 0.000 claims abstract description 203
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 94
- 238000012937 correction Methods 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 18
- 238000005259 measurement Methods 0.000 claims description 12
- 230000007704 transition Effects 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention provides a target tracking processing method and a target tracking processing device, wherein the method comprises the following steps: performing target detection on a current video frame of video data to obtain a target detection result, wherein the target detection result comprises target characteristics and a current motion state of the target; matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked; under the condition that the matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target; the target equipment for tracking the target in the next video frame is determined according to the estimated motion state, so that the problem that the tracking range is limited because the target searching is performed by expanding the searching range from the current position according to a certain proportion and searching is more time-consuming when the corresponding target moves at a higher speed in the related art can be solved, the same target can be tracked among a plurality of camera equipment, and the target tracking range is expanded.
Description
Technical Field
The invention relates to the field of image processing, in particular to a target tracking processing method and device.
Background
In the field of monitoring, a target is tracked, and the motion state of the target is tracked by a single camera in most cases. The prior art can generally work on the following principles: firstly, detecting a tracking target in each frame, then determining whether targets in the frames are identical or not through a specific technology, and finally determining the track of the targets. The object detection technology can be RCNN, yolo, etc., and the object association is to determine whether the tracked objects in different frames are the same or not according to the feature similarity of the objects. These techniques have some drawbacks, such as high complexity, long time consumption, poor generalization capability, and the like, and cannot meet various requirements in practical situations.
In the related art, a target tracking method is proposed, in which a neural network is used to detect a target characteristic in a target frame and perform similarity comparison with a given target characteristic, and the target is considered to be successfully tracked when the similarity is relatively higher than a certain threshold. When searching the target, the searching range is gradually enlarged according to a certain proportion to search the target. The target tracking of the single camera is mainly performed, but the defect is that the target searching is performed by expanding the searching range from the current position according to a certain proportion, the corresponding target moving speed is higher, the more time is consumed for searching, and the complexity of the equipment is increased to a certain extent.
Aiming at the problem that the tracking range is limited because the target is tracked based on a single camera in the related art, the target searching is to expand the searching range from the current position according to a certain proportion, and the faster the corresponding target moves, the more time is spent searching, and no solution is proposed.
Disclosure of Invention
The embodiment of the invention provides a target tracking processing method and device, which at least solve the problems that in the related art, a target is tracked based on a single camera, the target searching is to expand the searching range from the current position according to a certain proportion, the faster the corresponding target moving speed is, the more time is consumed, and the tracking range is limited.
According to an embodiment of the present invention, there is provided a target tracking processing method including:
performing target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises target characteristics and a current motion state of the target;
matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
Under the condition that the matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
and determining target equipment for tracking the target in the next video frame according to the estimated motion state.
Optionally, determining the estimated motion state of the target in the next video frame according to the current motion state of the target includes:
based on a Kalman motion estimation model, determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame;
And determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state includes:
the current motion state is corrected to obtain a current corrected motion state;
Determining an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by:
Wherein, For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
Optionally, the method further comprises:
determining a covariance matrix estimate of the state of the target in the next video frame by:
Wherein/> For covariance matrix estimation of the state of the target in the next video frame, P n-1 is covariance matrix estimation of the state of the target after correction in the current video frame, and Q n is covariance matrix of system noise in the next video frame;
And correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame, and obtaining the corrected motion state of the target in the next video frame includes:
acquiring an actual motion state of the target in the next video frame;
and correcting the estimated motion state according to the actual motion state of the target in the next video frame and covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
Optionally, the method further comprises:
correcting the estimated motion state according to the actual motion state of the target in the next video frame and covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame, wherein the estimated motion state is obtained by the following steps of:
Wherein x n is a corrected motion state of the target in the next video frame, z n is a target motion state observed by the target in the next video frame, S n is a target measurement cosine covariance matrix of the target in the next video frame, and R n is a covariance matrix of measurement noise of the target in the next video frame; k n is the kalman gain of the object in the next video frame, H n is the mapping matrix of the motion state of the object to the device observations.
Optionally, the method further comprises:
Correcting the covariance matrix estimate of the state of the target in the next video frame by the following method to obtain the covariance matrix estimate of the corrected state of the target in the next video frame:
Wherein P n is a covariance matrix estimate of the state of the target after correction in the next video frame.
Optionally, matching the characteristic information of the target with the characteristic information of the target to be tracked, which is predetermined, includes:
determining the similarity between the target characteristics and the characteristic information of a predetermined target to be tracked;
Under the condition that the similarity is larger than or equal to a preset threshold value, determining that the matching is successful;
And under the condition that the similarity is smaller than the preset threshold value, determining that the matching fails.
Optionally, determining the target device for tracking the target in the next video frame according to the estimated motion state includes:
determining a target monitoring area corresponding to the estimated motion state;
And determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
According to another embodiment of the present invention, there is also provided an object tracking processing apparatus including:
the target detection module is used for detecting the target of the current video frame of the video data to obtain a target detection result, wherein the detection result comprises the target characteristics and the current motion state of the target;
The matching module is used for matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
The first determining module is used for determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition that the matching is successful;
And the second determining module is used for determining target equipment for tracking the target in the next video frame according to the estimated motion state.
Optionally, the first determining module includes:
The first determining submodule is used for determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame based on a Kalman motion estimation model;
And the second determining submodule is used for determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, the second determining submodule includes:
The correcting unit is used for correcting the current motion state to obtain a current corrected motion state;
A determining unit, configured to determine an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by:
Wherein, For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
Optionally, the apparatus further comprises:
A third determination module for determining a covariance matrix estimate of the state of the target in the next video frame by:
Wherein/> For covariance matrix estimation of the state of the target in the next video frame, P n-1 is covariance matrix estimation of the state of the target after correction in the current video frame, and Q n is covariance matrix of system noise in the next video frame;
And the correction module is used for correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the correction module includes:
The acquisition sub-module is used for acquiring the actual motion state of the target in the next video frame;
And the first correction sub-module is used for correcting the estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
Optionally, the first correction submodule is further configured to correct the estimated motion state according to a covariance matrix estimate of an actual motion state of the target in the next video frame and a state of the target in the next video frame, so as to obtain a corrected motion state of the target in the next video frame:
Wherein x n is a corrected motion state of the target in the next video frame, z n is a target motion state observed by the target in the next video frame, S n is a target measurement cosine covariance matrix of the target in the next video frame, and R n is a covariance matrix of measurement noise of the target in the next video frame; k n is the kalman gain of the object in the next video frame, H n is the mapping matrix of the motion state of the object to the device observations.
Optionally, the apparatus further comprises:
A second correction sub-module, configured to correct the covariance matrix estimate of the state of the target in the next video frame by:
Wherein P n is a covariance matrix estimate of the state of the target after correction in the next video frame.
Optionally, the matching module includes:
a third determining submodule, configured to determine similarity between the target feature and feature information of a predetermined target to be tracked;
A fourth determining submodule, configured to determine that matching is successful when the similarity is greater than or equal to a preset threshold;
and a fifth determining submodule, configured to determine that matching fails if the similarity is smaller than the preset threshold.
Optionally, the second determining module is further configured to
Determining a target monitoring area corresponding to the estimated motion state;
And determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
According to a further embodiment of the invention, there is also provided a computer-readable storage medium having stored therein a computer program, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
According to a further embodiment of the invention, there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
According to the invention, when the characteristic information of the collected target is matched with the characteristic information of the target to be tracked in a plurality of cameras, the estimated motion state of the target in the next video frame is determined according to the current motion state of the target, the target equipment for tracking the target is determined according to the estimated motion state, and the target equipment tracks the target in the next video frame, so that the problem that the target tracking range is limited due to the fact that the target searching is expanded according to a certain proportion from the current position in the related art, the faster the corresponding target moving speed, the more time is consumed, and the problem that the tracking range is limited is solved, and the same target can be tracked among a plurality of camera equipment when the target moves among the camera equipment, and the target tracking range is expanded.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a block diagram of a hardware configuration of a mobile terminal of a target tracking processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a target tracking processing method according to an embodiment of the invention;
FIG. 3 is a flow chart of object tracking during movement of a tracked object in accordance with an embodiment of the present invention;
FIG. 4 is a flow chart of an implementation of a target tracking algorithm according to an embodiment of the invention;
Fig. 5 is a block diagram of a target tracking processing apparatus according to an embodiment of the present invention.
Detailed Description
The application will be described in detail hereinafter with reference to the drawings in conjunction with embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
Example 1
The method according to the first embodiment of the present application may be implemented in a mobile terminal, a computer terminal or a similar computing device. Taking a mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of the mobile terminal according to the target tracking processing method of the embodiment of the present application, as shown in fig. 1, the mobile terminal may include one or more (only one is shown in fig. 1) processors 102 (the processors 102 may include, but are not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, and optionally, the mobile terminal may further include a transmission device 106 for a communication function and an input/output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.
The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to the object tracking processing method in the embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, implement the above-mentioned method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as a NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet wirelessly.
In this embodiment, a target tracking processing method running on the mobile terminal or the network architecture is provided, fig. 2 is a flowchart of the target tracking processing method according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps:
Step S202, performing target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises target characteristics and a current motion state of the target;
the motion state in the embodiment of the invention at least comprises the speed and the position, namely the current motion state also at least comprises the speed and the position of the target.
Step S204, matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
In the embodiment of the present invention, the step S204 may specifically include: determining the similarity between the target characteristics and the characteristic information of a predetermined target to be tracked; under the condition that the similarity is larger than or equal to a preset threshold value, determining that the matching is successful; and under the condition that the similarity is smaller than the preset threshold value, determining that the matching fails.
Step S206, under the condition that the matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
Step S208, determining target equipment for tracking the target in the next video frame according to the estimated motion state.
In the embodiment of the present invention, the step S208 may specifically include: determining a target monitoring area corresponding to the estimated motion state; and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
Through the steps S202 to S208, when the characteristic information of the collected target is matched with the characteristic information of the target to be tracked in the plurality of cameras, the estimated motion state of the target in the next video frame is determined according to the current motion state of the target, the target equipment for tracking the target is determined according to the estimated motion state, and the target equipment tracks the target in the video frame at the next moment, so that the problem that the target tracking range is limited due to the fact that the target searching is expanded from the current position according to a certain proportion, the faster the searching of the corresponding target is, the problem that the tracking range is limited is solved, and the same target can be tracked among a plurality of camera equipment when the target moves between the camera equipment, and the target tracking range is expanded.
In the embodiment of the present invention, the step S206 may specifically include: based on a Kalman motion estimation model, determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame; determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state, and further correcting the current motion state to obtain a current corrected motion state; determining an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by: Wherein/> For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
In an alternative embodiment, the covariance matrix estimate of the state of the target in the next video frame is determined by: Wherein/> For covariance matrix estimation of the state of the target in the next video frame, P n-1 is covariance matrix estimation of the state of the target after correction in the current video frame, and Q n is covariance matrix of system noise in the next video frame; correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain a corrected motion state of the target in the next video frame, and further, obtaining an actual motion state of the target in the next video frame; correcting the estimated motion state according to the actual motion state of the target in the next video frame and covariance matrix estimation of the state of the target in the next video frame to obtain a corrected motion state of the target in the next video frame, and further obtaining the corrected motion state of the target in the next video frame by the following steps:
Wherein x n is a corrected motion state of the target in the next video frame, z n is a target motion state observed by the target in the next video frame, S n is a target measurement cosine covariance matrix of the target in the next video frame, and R n is a covariance matrix of measurement noise of the target in the next video frame; k n is the kalman gain of the object in the next video frame, H n is the mapping matrix of the motion state of the object to the device observations.
In another alternative embodiment, the covariance matrix estimate of the state of the object in the next video frame is obtained by modifying the covariance matrix estimate of the state of the object in the next video frame by: Wherein P n is a covariance matrix estimate of the state of the target after correction in the next video frame.
In the embodiment of the invention, each image pickup device is connected to the same central controller, and the storage device at the controller side stores video according to the tracked target. FIG. 3 is a flow chart of object tracking during movement of a tracked object, as shown in FIG. 3, according to an embodiment of the invention, including:
step S301, a source device tracks a target;
step S302, judging whether to switch the video equipment of the tracking target, if yes, executing step S303, otherwise returning to step S301;
step S303, the source device sends the characteristic information of the target to the target video device;
In step S304, the target video device acquires information of the target and tracks the target.
After the target is captured in one video device (source device), tracking according to a certain tracking algorithm, and if the tracked target leaves the monitoring field of the current camera and enters the monitoring field of other related devices (target video device), switching the target; otherwise, the target tracking is ended. After the target is switched, the source device sends the related information (such as position information) of the tracked target to the target device through the central controller, and then the target device tracks the tracked target. Thus, the target tracking video stored in the central controller will switch from the source device to the target device.
FIG. 4 is a flowchart of an implementation of a target tracking algorithm according to an embodiment of the invention, as shown in FIG. 4, including:
step S401, acquiring a current video frame acquired by video equipment;
step S402, detecting the target of the current video frame to obtain a target to be tracked;
Step S403, estimating the motion state of the target in the next video frame by adopting a Kalman motion model to obtain an estimated motion state;
step S404, determining a target video device according to the target according to the estimated motion state, and sending the target characteristics of the target to the target video device.
When a target is tracked in a given device, a frame of picture to be processed is firstly obtained from the camera device, then target detection is carried out on the picture, the tracked target is matched according to a target matching algorithm, and then the next occurrence position of the next frame of target is predicted by adopting a Kalman motion model according to the history and the current position of the target. Finally, the target is tracked jointly according to the future position of the target and the monitoring field of view of the associated camera device.
The target tracking algorithm in the embodiment of the invention mainly comprises the following steps:
Video input, video needs to be input according to a certain frame interval.
Target detection, target detection is performed by using a neural network, such as YoloV. The input of the network is a given area in the picture, and the size of the area needs to be specifically determined by the target detection network; the center of the region is determined by a kalman motion estimation model.
And after the objects are detected, extracting the characteristics of each object through a deep neural network, wherein the extracted characteristics can avoid the influence of factors such as light, visual angle, image noise interference and the like on the object.
And (3) matching the target depth characteristics to be tracked and the characteristics extracted from the object in the step (3), wherein the similarity can be a cosine similarity matrix, a Euclidean distance matrix and the like. If there are multiple tracking targets in the same device, the system uses the Hungarian algorithm to obtain the best match.
And (3) carrying out Kalman motion estimation, namely after detecting the target through target matching, predicting the position of the next frame of the target according to the current position and the historical position of the target.
Parameter updating, wherein the Kalman parameters are required to be updated according to the current position of the target after Kalman estimation; if the tracking target moves from the current device monitoring area to the video monitoring area of another device, the position parameter and the Kalman parameter of the target need to be updated to the other device through affine transformation.
The kalman motion estimation consists of two phases, namely a prediction phase and a parameter correction phase, which can be expressed in the following form:
prediction stage:
Wherein, Representing an estimation matrix of the state of motion (including position, velocity, etc.) of the target at time n,/>Representing covariance matrix estimation of the target at time n; f n denotes a state transition matrix of the image pickup apparatus at time n; x n-1 represents the motion state of the target at time n-1; b n represents the state control matrix of the target at time n, u n represents the state control vector of the target at time n; p n-1 represents the actual covariance matrix of the target at time n-1; q n represents the covariance matrix of the system noise at time n.
And (3) correction:
Wherein z n represents the motion state of the target at time n, which is actually observed by the video device; h n represents a mapping matrix of target motion states to device observations; s n represents the target measurement cosine covariance matrix at time n; r n denotes the covariance matrix of the measurement noise at time n; k n denotes the kalman gain at time n target.
The iterative process of motion estimation by using Kalman by the video device comprises the following steps:
The video equipment adopts a model estimation of a Kalman prediction stage to obtain a motion state matrix and a covariance matrix at a moment n according to the current position of the target and the historical position of the target;
Calculating to obtain the Gaussian distribution of the target operation motion state at the next moment: further, according to the result, obtaining the expected motion state of the target;
The motion state z n of the target at the time n is observed and obtained, and then the motion state x n of the target and the covariance matrix P n are updated according to the formula of the correction phase of the kalman motion estimation.
If the object moves linearly, the motion state x n is composed of position and velocity, and since the position can be represented by a point, x n can be represented as a 2x 1-dimensional vector, i.e., x n＝[pn,vn]T. Since the interval between processing frames by the video device is short, during this period, the target can be considered as uniformly accelerating, uniformly decelerating or uniformly moving, and then there are:
the motion state matrix at time 0 can refer to the actual situation, for example, the motion state matrix is set as follows: The covariance matrix of the time 0 state may be set to/> Q n and R n can be assumed to be a standard covariance distribution of 2 x 2; h n can consider the result obtained by the video device, i.e., the actually required result, to be a 2 x 2 identity matrix. According to the above assumption, an iterative process of kalman motion estimation is performed. It should be noted that u n cannot be obtained directly by observation, and can be obtained by performing approximate calculation through the change of the position of the target in the history frame.
Expanding to a two-dimensional plane to perform motion estimation on the target, the motion state x n is a vector of 4×1 dimensions, namely x n＝[px,n,py,n,vx,n,vy,n]T.
According to the embodiment of the invention, the deep reinforcement learning is directly applied to target tracking, so that the characteristics of long time consumption and low speed of many traditional algorithms are abandoned; in the invention, a Kalman motion model is adopted in a camera for target tracking of a target track to improve the speed; the characteristic extraction adopts a deep learning method, so that the speed is high; when the targets move among the image pickup devices, the same target can be tracked among the plurality of image pickup devices by adopting the radiation change, so that the target tracking range is enlarged, a plurality of targets can be tracked in the plurality of devices at the same time, and the target tracking has universality.
Example 2
According to another embodiment of the present invention, there is also provided an object tracking processing apparatus, fig. 5 is a block diagram of the object tracking processing apparatus according to an embodiment of the present invention, as shown in fig. 5, including:
The target detection module 52 is configured to perform target detection on a current video frame of video data to obtain a detection result of a target, where the detection result includes a target feature and a current motion state of the target;
a matching module 54, configured to match the feature information of the target with predetermined feature information of a target to be tracked;
a first determining module 56, configured to determine, if the matching is successful, an estimated motion state of the target in a next video frame according to a current motion state of the target;
A second determining module 58 is configured to determine a target device tracking the target in the next video frame according to the estimated motion state.
Optionally, the first determining module 56 includes:
The first determining submodule is used for determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame based on a Kalman motion estimation model;
And the second determining submodule is used for determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, the second determining submodule includes:
The correcting unit is used for correcting the current motion state to obtain a current corrected motion state;
A determining unit, configured to determine an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by:
Wherein, For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
Optionally, the apparatus further comprises:
A third determination module for determining a covariance matrix estimate of the state of the target in the next video frame by:
Wherein/> For covariance matrix estimation of the state of the target in the next video frame, P n-1 is covariance matrix estimation of the state of the target after correction in the current video frame, and Q n is covariance matrix of system noise in the next video frame;
And the correction module is used for correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the correction module includes:
The acquisition sub-module is used for acquiring the actual motion state of the target in the next video frame;
And the first correction sub-module is used for correcting the estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
Optionally, the first correction submodule is further configured to correct the estimated motion state according to a covariance matrix estimate of an actual motion state of the target in the next video frame and a state of the target in the next video frame, so as to obtain a corrected motion state of the target in the next video frame:
Wherein x n is a corrected motion state of the target in the next video frame, z n is a target motion state observed by the target in the next video frame, S n is a target measurement cosine covariance matrix of the target in the next video frame, and R n is a covariance matrix of measurement noise of the target in the next video frame; k n is the kalman gain of the object in the next video frame, H n is the mapping matrix of the motion state of the object to the device observations.
Optionally, the apparatus further comprises:
A second correction sub-module, configured to correct the covariance matrix estimate of the state of the target in the next video frame by:
Wherein P n is a covariance matrix estimate of the state of the target after correction in the next video frame.
Optionally, the matching module 54 includes:
a third determining submodule, configured to determine similarity between the target feature and feature information of a predetermined target to be tracked;
A fourth determining submodule, configured to determine that matching is successful when the similarity is greater than or equal to a preset threshold;
and a fifth determining submodule, configured to determine that matching fails if the similarity is smaller than the preset threshold.
Optionally, the second determining module 58 is further configured to
Determining a target monitoring area corresponding to the estimated motion state;
And determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; or the above modules may be located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a computer readable storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
Alternatively, in the present embodiment, the above-described storage medium may be configured to store a computer program for performing the steps of:
S1, performing target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises target characteristics and a current motion state of the target;
S2, matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
s3, under the condition that matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
S4, determining target equipment for tracking the target in the next video frame according to the estimated motion state.
Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
Example 4
An embodiment of the invention also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
S1, performing target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises target characteristics and a current motion state of the target;
S2, matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
s3, under the condition that matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
S4, determining target equipment for tracking the target in the next video frame according to the estimated motion state.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments and optional implementations, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A target tracking processing method, characterized by comprising:
performing target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises target characteristics and a current motion state of the target;
matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
Under the condition that the matching is successful, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
Determining target equipment for tracking the target in the next video frame according to the estimated motion state;
wherein determining the estimated motion state of the target in the next video frame according to the current motion state of the target comprises:
based on a Kalman motion estimation model, determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame;
determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state comprises the following steps: the current motion state is corrected to obtain a current corrected motion state; determining an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by: Wherein/> For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
2. The method according to claim 1, wherein the method further comprises:
determining a covariance matrix estimate of the state of the target in the next video frame by:
Wherein/> For covariance matrix estimation of the state of the target in the next video frame, P n-1 is covariance matrix estimation of the state of the target after correction in the current video frame, and Q n is covariance matrix of system noise in the next video frame;
And correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
3. The method of claim 2, wherein correcting the estimated motion state based on covariance matrix estimation of the state of the object in the next video frame, comprises:
acquiring an actual motion state of the target in the next video frame;
and correcting the estimated motion state according to the actual motion state of the target in the next video frame and covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
4. A method according to claim 3, characterized in that the method further comprises:
correcting the estimated motion state according to the actual motion state of the target in the next video frame and covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame, wherein the estimated motion state is obtained by the following steps of:
Wherein x n is a corrected motion state of the target in the next video frame, z n is a target motion state observed by the target in the next video frame, S n is a target measurement cosine covariance matrix of the target in the next video frame, and R n is a covariance matrix of measurement noise of the target in the next video frame; k n is the kalman gain of the object in the next video frame, H n is the mapping matrix of the motion state of the object to the device observations.
5. The method according to claim 4, wherein the method further comprises:
Correcting the covariance matrix estimate of the state of the target in the next video frame by the following method to obtain the covariance matrix estimate of the corrected state of the target in the next video frame:
Wherein P n is a covariance matrix estimate of the state of the target after correction in the next video frame.
6. The method according to any one of claims 1 to 5, wherein matching the characteristic information of the object with the predetermined characteristic information of the object to be tracked comprises:
determining the similarity between the target characteristics and the characteristic information of a predetermined target to be tracked;
Under the condition that the similarity is larger than or equal to a preset threshold value, determining that the matching is successful;
And under the condition that the similarity is smaller than the preset threshold value, determining that the matching fails.
7. The method of claim 6, wherein determining a target device in the next video frame that tracks the target based on the estimated motion state comprises:
determining a target monitoring area corresponding to the estimated motion state;
And determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
8. A target tracking processing apparatus, comprising:
the target detection module is used for detecting the target of the current video frame of the video data to obtain a target detection result, wherein the detection result comprises the target characteristics and the current motion state of the target;
The matching module is used for matching the characteristic information of the target with the characteristic information of a predetermined target to be tracked;
The first determining module is used for determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition that the matching is successful;
the second determining module is used for determining target equipment for tracking the target in the next video frame according to the estimated motion state;
Wherein the first determining module includes:
The first determining submodule is used for determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame based on a Kalman motion estimation model;
the second determining submodule is used for determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state;
wherein the second determining submodule includes:
The correcting unit is used for correcting the current motion state to obtain a current corrected motion state;
A determining unit, configured to determine an estimated motion state of the target in the next video frame according to the state control vector and the current corrected motion state by:
Wherein, For the estimated motion state, F n is a state transition matrix of the target in the next video frame, x n-1 is the current corrected motion state, u n is the state control vector, and B n is a state control matrix of the target in the next video frame.
9. A computer-readable storage medium, characterized in that the storage medium has stored therein a computer program, wherein the computer program is arranged to execute the method of any of the claims 1 to 7 when run.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010753234.9A CN112001948B (en) | 2020-07-30 | 2020-07-30 | Target tracking processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010753234.9A CN112001948B (en) | 2020-07-30 | 2020-07-30 | Target tracking processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112001948A CN112001948A (en) | 2020-11-27 |
| CN112001948B true CN112001948B (en) | 2024-06-11 |
Family
ID=73463320
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010753234.9A Active CN112001948B (en) | 2020-07-30 | 2020-07-30 | Target tracking processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112001948B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113092807B (en) * | 2021-04-21 | 2024-05-14 | 上海浦江桥隧运营管理有限公司 | Urban overhead road vehicle speed measuring method based on multi-target tracking algorithm |
| CN113344966B (en) * | 2021-06-18 | 2023-09-22 | 北京星航科创技术有限公司 | Target motion state correction method of motion camera |
| CN113628251B (en) * | 2021-10-11 | 2022-02-01 | 北京中科金马科技股份有限公司 | Smart hotel terminal monitoring method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783020A (en) * | 2010-03-04 | 2010-07-21 | 湖南大学 | Video multi-target fast tracking method based on joint probability data association |
| CN106780542A (en) * | 2016-12-29 | 2017-05-31 | 北京理工大学 | A kind of machine fish tracking of the Camshift based on embedded Kalman filter |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
| WO2019064375A1 (en) * | 2017-09-27 | 2019-04-04 | 日本電気株式会社 | Information processing device, control method, and program |
| CN109712173A (en) * | 2018-12-05 | 2019-05-03 | 北京空间机电研究所 | A kind of picture position method for estimating based on Kalman filter |
| CN109785368A (en) * | 2017-11-13 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of method for tracking target and device |
| CN110110649A (en) * | 2019-05-02 | 2019-08-09 | 西安电子科技大学 | Alternative method for detecting human face based on directional velocity |
| CN111145214A (en) * | 2019-12-17 | 2020-05-12 | 深圳云天励飞技术有限公司 | Target tracking method, device, terminal equipment and medium |
| CN111192296A (en) * | 2019-12-30 | 2020-05-22 | 长沙品先信息技术有限公司 | Pedestrian multi-target detection and tracking method based on video monitoring |
-
2020
- 2020-07-30 CN CN202010753234.9A patent/CN112001948B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783020A (en) * | 2010-03-04 | 2010-07-21 | 湖南大学 | Video multi-target fast tracking method based on joint probability data association |
| CN106780542A (en) * | 2016-12-29 | 2017-05-31 | 北京理工大学 | A kind of machine fish tracking of the Camshift based on embedded Kalman filter |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
| WO2019064375A1 (en) * | 2017-09-27 | 2019-04-04 | 日本電気株式会社 | Information processing device, control method, and program |
| CN109785368A (en) * | 2017-11-13 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of method for tracking target and device |
| CN109712173A (en) * | 2018-12-05 | 2019-05-03 | 北京空间机电研究所 | A kind of picture position method for estimating based on Kalman filter |
| CN110110649A (en) * | 2019-05-02 | 2019-08-09 | 西安电子科技大学 | Alternative method for detecting human face based on directional velocity |
| CN111145214A (en) * | 2019-12-17 | 2020-05-12 | 深圳云天励飞技术有限公司 | Target tracking method, device, terminal equipment and medium |
| CN111192296A (en) * | 2019-12-30 | 2020-05-22 | 长沙品先信息技术有限公司 | Pedestrian multi-target detection and tracking method based on video monitoring |
Non-Patent Citations (4)
| Title |
|---|
| 交通视频中的Kalman滤波的多车辆跟踪算法;贲晛烨;王科俊;李阳;谢滨姣;;应用科技(03);全文 * |
| 结合SURF与Kalman滤波的CAMShift跟踪算法;张磊;彭力;;电子测量与仪器学报(03);全文 * |
| 视频监视中运动目标的检测与跟踪算法;郑江滨, 张艳宁, 冯大淦, 赵荣椿;系统工程与电子技术(10);全文 * |
| 贲晛烨 ; 王科俊 ; 李阳 ; 谢滨姣 ; .交通视频中的Kalman滤波的多车辆跟踪算法.应用科技.2011,(03),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112001948A (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112001948B (en) | Target tracking processing method and device | |
| US20200364882A1 (en) | Method and apparatuses for target tracking, and storage medium | |
| US9852511B2 (en) | Systems and methods for tracking and detecting a target object | |
| CN109035304B (en) | Target tracking method, medium, computing device and apparatus | |
| US9760791B2 (en) | Method and system for object tracking | |
| CN107886048A (en) | Method for tracking target and system, storage medium and electric terminal | |
| CN110610150B (en) | Tracking method, device, computing equipment and medium of target moving object | |
| CN111209978B (en) | Three-dimensional visual repositioning method and device, computing equipment and storage medium | |
| CN108038415B (en) | Unmanned aerial vehicle automatic detection and tracking method based on machine vision | |
| CN111512317A (en) | Multi-target real-time tracking method and device and electronic equipment | |
| CN109035295B (en) | Multi-target tracking method, device, computer equipment and storage medium | |
| CA2910965A1 (en) | Tracker assisted image capture | |
| CN104517275A (en) | Object detection method and system | |
| EP3531340B1 (en) | Human body tracing method, apparatus and device, and storage medium | |
| CN113191180A (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN112150514A (en) | Pedestrian trajectory tracking method, device and equipment of video and storage medium | |
| CN113112542A (en) | Visual positioning method and device, electronic equipment and storage medium | |
| CN116645396A (en) | Track determination method, track determination device, computer-readable storage medium and electronic device | |
| CN115063454A (en) | Multi-target tracking matching method, device, terminal and storage medium | |
| US20220366691A1 (en) | Object detection | |
| CN111665490B (en) | Target tracking method and device, storage medium and electronic device | |
| CN111507999B (en) | Target tracking method and device based on FDSST algorithm | |
| KR101595334B1 (en) | Method and apparatus for movement trajectory tracking of moving object on animal farm | |
| CN111652907A (en) | Multi-target tracking method and device based on data association and electronic equipment | |
| CN112396630A (en) | Method and device for determining state of target object, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |