CN112330714B - Pedestrian tracking method and device, electronic equipment and storage medium - Google Patents
Pedestrian tracking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112330714B CN112330714B CN202011053262.6A CN202011053262A CN112330714B CN 112330714 B CN112330714 B CN 112330714B CN 202011053262 A CN202011053262 A CN 202011053262A CN 112330714 B CN112330714 B CN 112330714B
- Authority
- CN
- China
- Prior art keywords
- tracking
- joint
- gesture
- information
- attitude
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000012544 monitoring process Methods 0.000 claims abstract description 54
- 230000001629 suppression Effects 0.000 claims abstract description 20
- 239000013598 vector Substances 0.000 claims description 29
- 238000001514 detection method Methods 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 22
- 238000005259 measurement Methods 0.000 claims description 9
- 230000007704 transition Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 11
- 210000001503 joint Anatomy 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 210000002310 elbow joint Anatomy 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 210000000629 knee joint Anatomy 0.000 description 2
- 210000000323 shoulder joint Anatomy 0.000 description 2
- 101100289792 Squirrel monkey polyomavirus large T gene Proteins 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The application provides a pedestrian tracking method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a monitoring video in a preset monitoring area; extracting the gesture joint information of each tracking object in the monitoring area in the monitoring video; judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm; when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule; and determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object. By estimating the joint information of the blocked joint of the blocked object to obtain the attitude joint estimation information, the tracking result of the blocked object is further determined, the robustness of the pedestrian tracking method is improved, and the accuracy of the tracking result is improved.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a pedestrian tracking method, a device, an electronic device, and a storage medium.
Background
At present, with the continuous development of artificial intelligence technology, pedestrian tracking technology has been gradually applied to the fields of monitoring security protection and the like. For areas such as road bayonets, shops and stores, people can be monitored by arranging a monitoring camera, and all pedestrians in and out of a monitoring image are continuously detected by utilizing a pedestrian tracking technology, so that the same pedestrian is continuously tracked.
In the prior art, a pedestrian frame is usually extracted from a monitoring image, and feature vectors of the pedestrian frames are extracted by using a convolutional neural network multi-hypothesis method, a minimum cost flow method, a minimum multi-segmentation method and other traditional methods. And matching the pedestrian frames in all the monitoring images based on the characteristic vector of each pedestrian frame so as to obtain a pedestrian tracking result.
However, when the pedestrian is blocked by other objects, if pedestrian tracking is performed based on the prior art, the situation that the extracted feature vector of the pedestrian frame cannot be successfully matched with the existing pedestrian frame in the system occurs, so that the tracking result is output in an interrupted or erroneous tracking result is output. Therefore, a pedestrian tracking method with high robustness is urgently needed, and the method has important significance for improving the accuracy of a tracking result.
Disclosure of Invention
The application provides a pedestrian tracking method, a device, electronic equipment and a storage medium, which are used for solving the defects of poor robustness and the like of the pedestrian tracking method in the prior art.
A first aspect of the present application provides a pedestrian tracking method, including:
acquiring a monitoring video in a preset monitoring area;
extracting the gesture joint information of each tracking object in the monitoring area from the monitoring video; wherein the gesture joint information comprises the position of each joint of the tracking object and the movement speed of each joint;
judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm;
when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule;
and determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object.
Optionally, the method further comprises:
and when the tracking object is determined not to be an occlusion object, determining a tracking result of the tracking object according to the gesture joint information of the tracking object.
Optionally, the determining the tracking result of the blocked object according to the pose joint information and the pose joint estimation information of the blocked object includes:
generating a corresponding attitude state vector according to the attitude joint information and the attitude joint estimation information of the blocked object;
and inputting the state vector to a preset Kalman filter to obtain a tracking result of the shielded object.
Optionally, the determining, based on a preset non-maximum suppression algorithm, whether each tracking object is an occluded object according to the gesture joint information of each tracking object includes:
calculating the gesture detection confidence of each tracking object according to the gesture joint information of each tracking object based on the non-maximum suppression algorithm;
judging whether the gesture detection confidence of each tracking object belongs to a preset confidence interval or not;
and when the gesture detection confidence of the tracking object is not determined to be in the confidence interval, determining that the tracking object is an occluded object.
Optionally, the method further comprises:
and when determining that the gesture detection confidence of the tracking object belongs to the confidence interval, determining that the tracking object is not an occluded object.
A second aspect of the present application provides a pedestrian tracking apparatus comprising:
the acquisition module is used for acquiring the monitoring video in the preset monitoring area;
the extraction module is used for extracting the gesture joint information of each tracking object in the monitoring area from the monitoring video; wherein the gesture joint information comprises the position of each joint of the tracking object and the movement speed of each joint;
the judging module is used for judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum value suppression algorithm;
the estimating module is used for estimating the attitude joint estimating information of the blocked object according to a preset attitude joint information association rule when the tracking object is determined to be the blocked object;
and the tracking module is used for determining a tracking result of the shielded object according to the gesture joint information and the gesture joint estimation information of the shielded object.
Optionally, the tracking module is further configured to:
and when the tracking object is determined not to be an occlusion object, determining a tracking result of the tracking object according to the gesture joint information of the tracking object.
Optionally, the tracking module is specifically configured to:
generating a corresponding attitude state vector according to the attitude joint information and the attitude joint estimation information of the blocked object;
and inputting the state vector to a preset Kalman filter to obtain a tracking result of the shielded object.
Optionally, the judging module is specifically configured to:
calculating the gesture detection confidence of each tracking object according to the gesture joint information of each tracking object based on the non-maximum suppression algorithm;
judging whether the gesture detection confidence of each tracking object belongs to a preset confidence interval or not;
and when the gesture detection confidence of the tracking object is not determined to be in the confidence interval, determining that the tracking object is an occluded object.
Optionally, the estimation module is further configured to:
and when determining that the gesture detection confidence of the tracking object belongs to the confidence interval, determining that the tracking object is not an occluded object.
A third aspect of the present application provides a method comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes the computer-executable instructions stored by the memory such that the at least one processor performs the method as described above in the first aspect and the various possible designs of the first aspect.
A fourth aspect of the present application provides a computer-readable storage medium having stored therein computer-executable instructions which, when executed by a processor, implement the method as described above in the first aspect and the various possible designs of the first aspect.
The technical scheme of the application has the following advantages:
the pedestrian tracking method, the device, the electronic equipment and the storage medium provided by the application are used for acquiring the monitoring video in the preset monitoring area; extracting the gesture joint information of each tracking object in the monitoring area in the monitoring video; the gesture joint information comprises the positions of all joints of a tracking object and the movement speed of all joints; judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm; when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule; and determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object. The method provided by the scheme can track the blocked object, particularly, the joint information of the blocked joint of the blocked object is estimated to obtain the attitude joint estimation information, so that the tracking result is determined, the robustness of the pedestrian tracking method is improved, and the accuracy of the tracking result is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description will be given below of the drawings required for the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a schematic diagram of a pedestrian tracking system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a pedestrian tracking method according to an embodiment of the present application;
FIG. 3 is a flow chart of an exemplary pedestrian tracking method provided by an embodiment of the present application;
fig. 4 is a schematic structural diagram of a pedestrian tracking apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the disclosed concepts in any way, but to illustrate the concepts of the present application to those skilled in the art with reference to the specific embodiments.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
In the following description of the embodiments, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the prior art, a pedestrian frame is usually extracted from a monitoring image, and feature vectors of the pedestrian frames are extracted by using a convolutional neural network multi-hypothesis method, a minimum cost flow method, a minimum multi-segmentation method and other traditional methods. And matching the pedestrian frames in all the monitoring images based on the characteristic vector of each pedestrian frame so as to obtain a pedestrian tracking result. However, when the pedestrian is blocked by other objects, if pedestrian tracking is performed based on the prior art, the situation that the extracted feature vector of the pedestrian frame cannot be successfully matched with the existing pedestrian frame in the system occurs, so that the tracking result is output in an interrupted or erroneous tracking result is output.
Aiming at the problems, the pedestrian tracking method, the device, the electronic equipment and the storage medium provided by the embodiment of the application are used for acquiring the monitoring video in the preset monitoring area; extracting the gesture joint information of each tracking object in the monitoring area in the monitoring video; the gesture joint information comprises the positions of all joints of a tracking object and the movement speed of all joints; judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm; when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule; and determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object. The method provided by the scheme can track the blocked object, particularly, the joint information of the blocked joint of the blocked object is estimated to obtain the attitude joint estimation information, so that the tracking result is determined, the robustness of the pedestrian tracking method is improved, and the accuracy of the tracking result is improved.
The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
First, a description will be given of a structure of a pedestrian tracking system on which the present application is based:
the pedestrian tracking method, the device, the electronic equipment and the storage medium are suitable for tracking pedestrians in a preset monitoring area. Fig. 1 is a schematic structural diagram of a pedestrian tracking system according to an embodiment of the present application, which mainly includes a video capturing device and an electronic device, where a pedestrian tracking device for pedestrian tracking is disposed in the electronic device. Specifically, the video acquisition device is used for acquiring the monitoring video in the preset monitoring area, the acquired monitoring video is sent to the pedestrian tracking device, the pedestrian tracking device judges whether each tracking object is an occluded object, estimates the attitude joint estimation information of the occluded object according to the obtained attitude joint information aiming at the occluded object, and further determines the tracking result.
The embodiment of the application provides a pedestrian tracking method, which is used for tracking pedestrians in a preset monitoring area. The execution subject of the embodiment of the application is an electronic device, such as a server, a desktop computer, a notebook computer, a tablet computer, and other electronic devices that can be used for pedestrian tracking.
As shown in fig. 2, a flow chart of a pedestrian tracking method according to an embodiment of the present application is shown, where the method includes:
step 201, obtaining a monitoring video in a preset monitoring area.
It should be explained that the preset monitoring area may be set according to actual requirements, for example, when a pedestrian entering the park needs to be tracked, the preset monitoring area may be an entrance and an exit of the park.
Step 202, extracting the posture joint information of each tracking object in the monitoring area in the monitoring video.
Wherein the gesture joint information comprises the position of each joint of the tracked object and the movement speed of each joint.
It should be explained that the posture joint information mainly includes posture information corresponding to joints such as a neck joint, a left shoulder joint, a right shoulder joint, a left elbow joint, a right elbow joint, a left knee joint, and a right knee joint.
Specifically, the monitoring video is subjected to framing processing to obtain a plurality of monitoring images, human body characteristics in each monitoring image are extracted through identification of each monitoring image, and posture joint information of each tracking object is determined according to the extracted human body characteristics.
Step 203, based on a preset non-maximum suppression algorithm, judging whether each tracking object is an occluded object according to the posture joint information of each tracking object.
Specifically, in the suppression process of the non-maximum suppression algorithm, the blocked pedestrians can be screened out by means of the high confidence detection score of the non-blocked body part, wherein other suppression algorithms or other image recognition methods can be adopted to judge whether each tracking object is a blocked object, and the embodiment of the application is not limited.
Specifically, in an embodiment, the pose detection confidence of each tracking object may be calculated according to the pose joint information of each tracking object based on a non-maximum suppression algorithm; judging whether the gesture detection confidence of each tracking object belongs to a preset confidence interval; and when the gesture detection confidence of the tracking object is determined not to belong to the confidence interval, determining that the tracking object is an occluded object.
The confidence interval may be set according to actual situations, which is not limited in the embodiments of the present application.
Accordingly, in an embodiment, when it is determined that the pose detection confidence of the tracked object belongs to the confidence interval, it is determined that the tracked object is not an occluded object.
And 204, when the tracking object is determined to be the blocked object, estimating the attitude joint estimation information of the blocked object according to the preset attitude joint information association rule.
It should be explained that the posture joint information association rule is specifically a bipartite graph model between current detection and current tracking estimation, specifically, the problem of bipartite graph association can be solved by adopting a KM algorithm, and the posture change track of each joint is managed by using an association result.
Accordingly, in an embodiment, when it is determined that the tracked object is not an occluding object, a tracking result of the tracked object is determined according to the gesture joint information of the tracked object.
Step 205, determining the tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object.
Specifically, in an embodiment, a corresponding pose state vector may be generated according to pose joint information and pose joint estimation information of the occluded object; and inputting the state vector into a preset Kalman filter to obtain a tracking result of the shielded object.
By way of example, 16 pieces of attitude joint data in the attitude joint information and the attitude joint estimation information may be selected as state vectors of the kalman filter:
x＝{x 1 ，y 1 ，x 2 ，y 2 ，...，x 16 ，y 16 ，v x1 ，v y1 ，...，v x16 ，v y16 }
wherein x is i ，y i V is the position of the joint node of the gesture on the abscissa and the ordinate xi ，v yi The state transfer equation is as follows, which is the velocity component of the joint node in the transverse and longitudinal directions:
x t ＝A t|t-1 x x-1 +ω t-1
the observation equation is:
z t ＝Hx t +r t
wherein z is t Represents the observed state of the Kalman filter at the time t, x t And x t-1 Node states of the gesture joints at the time t and the time t-1 are corresponding. A is that t|t-1 Representing a state transition matrix, and transferring the motion position of each gesture joint node correspondingly; h is an observation matrix representing the result of measurement from the state vector after the prediction is completed, and the measurement vector is denoted as z. r is (r) t Corresponding to the measurement noise, obeying normal distribution, wherein the covariance is R; omega t-1 And corresponding to system noise, and also meeting normal distribution.
The state transition matrix for the pose articulation state is 64x32, with values of:
the observation matrix is thus a matrix of 32x 64:
at the filter initialization, we set 0 for the first observation for the velocity component in the state vector x. Since the target speed in the actual state may be any value, the speed state in the observed noise covariance matrix P in the kalman filter is set to a larger value corresponding to the initial variable.
The observed noise in the Kalman filter accords with normal distribution, so that the covariance matrix of the observed noise in the Kalman filter updating equation and the covariance matrix of the noise in the state transition process are set to be unit matrices.
Specifically, in one embodiment, the tracking trajectory may be established or deleted when the tracked object enters or leaves the scene. In the scene of entering the tracking object, a new tracking object can be considered to enter when the detection number is larger than the tracking number. There is also a case: the current frame has more than one tracked object disappeared at the same time and more than one tracked object appears for the first time. And therefore a new tracking object cannot be judged with a simple tracking object number. A tracking trajectory is established for a tracking object when a certain detected-to-predicted association distance is below a certain threshold and no tracking object is associated. Different association rules use different thresholds to decide whether a new tracked object is a new tracked object, wherein a minimum survival time T is set for the newly established tracked object in consideration of the possibility of false detection of the new tracked object f I.e. only the tracking object is subsequently successfully associated beyond T f Time, the tracker is formally established, otherwise, the tracking object is discarded. The track destruction strategy is to track the object over T d Frames will be destroyed when not associated. The strategy can enable the tracking object to be temporarily blocked, missed or detected to be far away from the correlation, and the position of the tracking object at the moment can be estimated by means of a Kalman filter only at T d And the time is re-associated, so that tracking can be kept. In practical application, a large T is set in a data set of sparse people d The value prevents anomalies from occurring, indicating that this is an effective strategy for handling occlusion. However, in dense scenes, if the tracking object is not destroyed immediately after disappearing, the tracking object is easily detected and correlated by other pedestrians, and T is hoped in the scene d The smaller the better.
As shown in fig. 3, an exemplary flow chart of a pedestrian tracking method provided in the embodiment of the present application is illustrated, where the flow chart illustrated in fig. 3 is a specific implementation manner of the pedestrian tracking method illustrated in fig. 2, and the two principles are the same and are not repeated.
According to the pedestrian tracking method, the monitoring video in the preset monitoring area is obtained; extracting the gesture joint information of each tracking object in the monitoring area in the monitoring video; the gesture joint information comprises the positions of all joints of a tracking object and the movement speed of all joints; judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm; when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule; and determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object. The method provided by the scheme can track the blocked object, particularly, the joint information of the blocked joint of the blocked object is estimated to obtain the attitude joint estimation information, so that the tracking result is determined, the robustness of the pedestrian tracking method is improved, and the accuracy of the tracking result is improved.
The embodiment of the application provides a pedestrian tracking device for executing the pedestrian tracking method provided by the embodiment.
Fig. 4 is a schematic structural diagram of a pedestrian tracking apparatus according to an embodiment of the present application. The apparatus 40 includes: an acquisition module 401, an extraction module 402, a judgment module 403, an estimation module 404 and a tracking module 405,
The acquiring module 401 is configured to acquire a surveillance video in a preset surveillance area; an extracting module 402, configured to extract, in a surveillance video, pose joint information of each tracked object in a surveillance area; the gesture joint information comprises the positions of all joints of a tracking object and the movement speed of all joints; a judging module 403, configured to judge whether each tracking object is an occluded object according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm; the estimating module 404 is configured to estimate pose joint estimation information of the blocked object according to a preset pose joint information association rule when the tracking object is determined to be the blocked object; and the tracking module 405 is configured to determine a tracking result of the blocked object according to the pose joint information and the pose joint estimation information of the blocked object.
Specifically, in an embodiment, the tracking module 405 is further configured to:
when the tracking object is determined not to be an occlusion object, determining a tracking result of the tracking object according to the gesture joint information of the tracking object.
Specifically, in one embodiment, the tracking module 405 is specifically configured to:
generating a corresponding attitude state vector according to the attitude joint information and the attitude joint estimation information of the blocked object;
and inputting the state vector into a preset Kalman filter to obtain a tracking result of the shielded object.
Specifically, in one embodiment, the determining module 403 is specifically configured to:
calculating the gesture detection confidence of each tracked object according to the gesture joint information of each tracked object based on a non-maximum suppression algorithm;
judging whether the gesture detection confidence of each tracking object belongs to a preset confidence interval;
and when the gesture detection confidence of the tracking object is determined not to belong to the confidence interval, determining that the tracking object is an occluded object.
Specifically, in an embodiment, the estimation module 404 is further configured to:
and when the gesture detection confidence of the tracking object is determined to belong to the confidence interval, determining that the tracking object is not an occluded object.
The specific manner in which the individual modules perform the operations in relation to the pedestrian tracking apparatus in this embodiment has been described in detail in relation to the embodiments of the method, and will not be described in detail here.
The pedestrian tracking device provided in the embodiment of the present application is configured to execute the pedestrian tracking method provided in the foregoing embodiment, and its implementation manner and principle are the same and are not described in detail.
The embodiment of the application provides electronic equipment for executing the pedestrian tracking method provided by the embodiment.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 50 includes: at least one processor 51 and a memory 52;
the memory stores computer-executable instructions; at least one processor executes computer-executable instructions stored in a memory, causing the at least one processor to perform the method as provided in any one of the embodiments above.
The embodiment of the application provides an electronic device, which is configured to execute the pedestrian tracking method provided by the embodiment, and its implementation manner and principle are the same and are not repeated.
The embodiment of the application provides a computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, and when a processor executes the computer executable instructions, the pedestrian tracking method provided by any embodiment is realized.
The storage medium including the computer executable instructions in the embodiments of the present application may be used to store the computer executable instructions of the pedestrian tracking method provided in the foregoing embodiments, and the implementation manner and principle of the storage medium are the same, and are not repeated.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be other manners of division when actually implemented.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above. The specific working process of the above-described device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims (8)
1. A pedestrian tracking method, comprising:
acquiring a monitoring video in a preset monitoring area;
extracting the gesture joint information of each tracking object in the monitoring area from the monitoring video; wherein the gesture joint information comprises the position of each joint of the tracking object and the movement speed of each joint;
judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum suppression algorithm;
when the tracking object is determined to be an occluded object, estimating the attitude joint estimation information of the occluded object according to a preset attitude joint information association rule;
determining a tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object;
the determining the tracking result of the blocked object according to the gesture joint information and the gesture joint estimation information of the blocked object comprises the following steps:
generating a corresponding attitude state vector according to the attitude joint information and the attitude joint estimation information of the blocked object;
inputting the attitude state vector to a preset Kalman filter to obtain a tracking result of the shielded object;
the step of inputting the attitude status vector to a preset kalman filter to obtain a tracking result of the shielded object includes:
the 16 pieces of attitude joint data in the attitude joint information and the attitude joint estimation information are selected as state vectors of the Kalman filter:
x＝{x 1 ，y 1 ，x 2 ，y 2 ，...，x 16 ，y 16 ，v x1 ，v y1 ，...，v x16 ，v y16 }
wherein x is i ，y i V is the position of the joint node of the gesture on the abscissa and the ordinate xi ，v yi The state transfer equation is as follows, which is the velocity component of the joint node in the transverse and longitudinal directions:
x t ＝A t|t-1 x x-1 +ω t-1
the observation equation is:
z t ＝Hx t +r t
wherein z is t Represents the observed state of the Kalman filter at the time t, x t And x t-1 Node states of posture joints corresponding to time t and time t-1, A t|t-1 Representing a state transition matrix, and transferring the motion position of each gesture joint node correspondingly; h is an observation matrix representing the result of measurement from the state vector after the prediction is completed, the measurement vector being denoted as z, r t Corresponding to the measurement noise, obeying normal distribution, wherein the covariance is R; omega t-1 Corresponding to system noise, and also meeting normal distribution;
the state transition matrix for the pose articulation state is 64x32, with values of:
the observation matrix is a matrix of 32x 64:
the observed noise in the Kalman filter accords with normal distribution, and the covariance matrix of the observed noise in the Kalman filter updating equation and the covariance matrix of the noise in the state transition process are set to be unit matrices.
2. The pedestrian tracking method according to claim 1, characterized by further comprising:
and when the tracking object is determined not to be an occlusion object, determining a tracking result of the tracking object according to the gesture joint information of the tracking object.
3. The pedestrian tracking method according to claim 1, wherein the determining whether each tracked object is an occluded object based on the posture joint information of each tracked object based on a preset non-maximum suppression algorithm includes:
calculating the gesture detection confidence of each tracking object according to the gesture joint information of each tracking object based on the non-maximum suppression algorithm;
judging whether the gesture detection confidence of each tracking object belongs to a preset confidence interval or not;
and when the gesture detection confidence of the tracking object is not determined to be in the confidence interval, determining that the tracking object is an occluded object.
4. A pedestrian tracking method as claimed in claim 3, further comprising:
and when determining that the gesture detection confidence of the tracking object belongs to the confidence interval, determining that the tracking object is not an occluded object.
5. A pedestrian tracking device, comprising:
the acquisition module is used for acquiring the monitoring video in the preset monitoring area;
the extraction module is used for extracting the gesture joint information of each tracking object in the monitoring area from the monitoring video; wherein the gesture joint information comprises the position of each joint of the tracking object and the movement speed of each joint;
the judging module is used for judging whether each tracking object is a blocked object or not according to the gesture joint information of each tracking object based on a preset non-maximum value suppression algorithm;
the estimating module is used for estimating the attitude joint estimating information of the blocked object according to a preset attitude joint information association rule when the tracking object is determined to be the blocked object;
the tracking module is used for determining a tracking result of the shielded object according to the gesture joint information and the gesture joint estimation information of the shielded object;
the tracking module is specifically configured to:
generating a corresponding attitude state vector according to the attitude joint information and the attitude joint estimation information of the blocked object;
inputting the attitude state vector to a preset Kalman filter to obtain a tracking result of the shielded object;
the tracking module is specifically configured to:
the 16 pieces of attitude joint data in the attitude joint information and the attitude joint estimation information are selected as state vectors of the Kalman filter:
x＝{x 1 ，y 1 ，x 2 ，y 2 ，...，x 16 ,y 16 ，v x1 ，v y1 ，...，v x16 ，v y16 }
wherein x is i ，y i V is the position of the joint node of the gesture on the abscissa and the ordinate xi ，v yi The state transfer equation is as follows, which is the velocity component of the joint node in the transverse and longitudinal directions:
x t ＝A t|t-1 x x-1 +ω t-1
the observation equation is:
z t ＝Hx t +r t
wherein z is t Represents the observed state of the Kalman filter at the time t, x t And x t-1 Node states of posture joints corresponding to time t and time t-1, A t|t-1 Representing a state transition matrix, and transferring the motion position of each gesture joint node correspondingly; h is an observation matrix representing the result of measurement from the state vector after the prediction is completed, the measurement vector being denoted as z, r t Corresponding to the measurement noise, obeying normal distribution, wherein the covariance is R; omega t-1 Corresponding to system noise, and also meeting normal distribution;
the state transition matrix for the pose articulation state is 64x32, with values of:
the observation matrix is a matrix of 32x 64:
the observed noise in the Kalman filter accords with normal distribution, and the covariance matrix of the observed noise in the Kalman filter updating equation and the covariance matrix of the noise in the state transition process are set to be unit matrices.
6. The pedestrian tracking device of claim 5, wherein the tracking module is further configured to:
and when the tracking object is determined not to be an occlusion object, determining a tracking result of the tracking object according to the gesture joint information of the tracking object.
7. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1 to 4.
8. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011053262.6A CN112330714B (en) | 2020-09-29 | 2020-09-29 | Pedestrian tracking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011053262.6A CN112330714B (en) | 2020-09-29 | 2020-09-29 | Pedestrian tracking method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112330714A CN112330714A (en) | 2021-02-05 |
| CN112330714B true CN112330714B (en) | 2024-01-09 |
Family
ID=74314171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011053262.6A Active CN112330714B (en) | 2020-09-29 | 2020-09-29 | Pedestrian tracking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330714B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107260179A (en) * | 2017-06-08 | 2017-10-20 | 朱翔 | Human body motion tracking method based on inertia and body-sensing sensing data quality evaluation |
| CN108629283A (en) * | 2018-04-02 | 2018-10-09 | 北京小米移动软件有限公司 | Face tracking method, device, equipment and storage medium |
| CN108734735A (en) * | 2017-04-17 | 2018-11-02 | 佳能株式会社 | Object shapes tracks of device and method and image processing system |
| CN110210302A (en) * | 2019-04-26 | 2019-09-06 | 平安科技（深圳）有限公司 | Multi-object tracking method, device, computer equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007310707A (en) * | 2006-05-19 | 2007-11-29 | Toshiba Corp | Apparatus and method for estimating posture |
| US9904852B2 (en) * | 2013-05-23 | 2018-02-27 | Sri International | Real-time object detection, tracking and occlusion reasoning |
| KR101933921B1 (en) * | 2013-06-03 | 2018-12-31 | 삼성전자주식회사 | Method and apparatus for estimating pose |
-
2020
- 2020-09-29 CN CN202011053262.6A patent/CN112330714B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734735A (en) * | 2017-04-17 | 2018-11-02 | 佳能株式会社 | Object shapes tracks of device and method and image processing system |
| CN107260179A (en) * | 2017-06-08 | 2017-10-20 | 朱翔 | Human body motion tracking method based on inertia and body-sensing sensing data quality evaluation |
| CN108629283A (en) * | 2018-04-02 | 2018-10-09 | 北京小米移动软件有限公司 | Face tracking method, device, equipment and storage medium |
| CN110210302A (en) * | 2019-04-26 | 2019-09-06 | 平安科技（深圳）有限公司 | Multi-object tracking method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112330714A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gabriel et al. | The state of the art in multiple object tracking under occlusion in video sequences | |
| US11580747B2 (en) | Multi-spatial scale analytics | |
| EP1844443B1 (en) | Classifying an object in a video frame | |
| Di Lascio et al. | A real time algorithm for people tracking using contextual reasoning | |
| KR102465960B1 (en) | Multi-Class Multi-Object Tracking Method using Changing Point Detection | |
| WO2009085233A2 (en) | System and method for visually tracking with occlusions | |
| EP1889205A2 (en) | Human detection and tracking for security applications | |
| Liu et al. | Moving object detection and tracking based on background subtraction | |
| CN113396423A (en) | Method of processing information from event-based sensors | |
| Nallasivam et al. | Moving human target detection and tracking in video frames | |
| Tang et al. | Hybrid blob and particle filter tracking approach for robust object tracking | |
| Mao et al. | Automated multiple target detection and tracking in UAV videos | |
| KR101492059B1 (en) | Real Time Object Tracking Method and System using the Mean-shift Algorithm | |
| CN112330714B (en) | Pedestrian tracking method and device, electronic equipment and storage medium | |
| Kausalya et al. | Idle object detection in video for banking ATM applications | |
| Galoogahi | Tracking groups of people in presence of occlusion | |
| Calderara et al. | A distributed outdoor video surveillance system for detection of abnormal people trajectories | |
| Sánchez et al. | Video tracking improvement using context-based information | |
| El Maadi et al. | Suspicious motion patterns detection and tracking in crowded scenes | |
| KR20160136157A (en) | Method and Apparatus for Segmenting Background Motion Region From Image | |
| Boufama et al. | Tracking multiple people in the context of video surveillance | |
| Avinash et al. | Automated video surveillance for retail store statistics generation | |
| Noceti et al. | Combined motion and appearance models for robust object tracking in real-time | |
| Belmouhcine et al. | Robust deep simple online real-time tracking | |
| Gupta et al. | Occlusion robust object tracking with modified particle filter framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |