CN106156705B - Pedestrian abnormal behavior detection method and system - Google Patents
Pedestrian abnormal behavior detection method and system Download PDFInfo
- Publication number
- CN106156705B CN106156705B CN201510160836.2A CN201510160836A CN106156705B CN 106156705 B CN106156705 B CN 106156705B CN 201510160836 A CN201510160836 A CN 201510160836A CN 106156705 B CN106156705 B CN 106156705B
- Authority
- CN
- China
- Prior art keywords
- target pedestrian
- pedestrian
- pedestrians
- video frame
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention relates to a pedestrian abnormal behavior detection method, which comprises the following steps: tracking a target pedestrian in a video frame by adopting a track segment association method; respectively calculating the movement distance of a target pedestrian in the video frame and the movement distance of the pedestrians around the target pedestrian in the video frame in the whole tracking process; and judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian and the pedestrians around the target pedestrian in the video frame in the whole tracking process. The invention also relates to a pedestrian abnormal behavior detection system. The pedestrian safety detection method and the pedestrian safety detection system can detect the wandering or lingering behavior of the pedestrian in the walking process, improve the efficiency of searching for the reason causing the safety problem by monitoring personnel, and save manpower.
Description
Technical Field
The invention relates to a method and a system for detecting abnormal behaviors of pedestrians.
Background
In recent years, as security issues are receiving increasing attention from society, detection of abnormal behavior in video is also becoming more important. Inconsistent with the behavior of surrounding pedestrians, wandering or lingering behavior exists, which may raise some safety concerns.
Through analyzing the surveillance video, some abnormal behaviors causing safety problems are judged, a large amount of information which is useless for security protection in the surveillance video can be filtered, and a large amount of manpower is saved.
At present, for the detection of abnormal behaviors of pedestrians, a target pedestrian is usually tracked to obtain a track of the target pedestrian, and the abnormal behaviors of the target pedestrian are detected through the consistency of the track and a scene model, or the abnormal behaviors are detected through the model.
Therefore, most of the existing pedestrian abnormal behavior detection modes need to establish a complex model and then carry out model learning, so that the efficiency is low and the process is complex.
Disclosure of Invention
In view of the above, it is desirable to provide a method and a system for detecting abnormal behavior of a pedestrian.
The invention provides a pedestrian abnormal behavior detection method, which comprises the following steps: a. tracking a target pedestrian in a video frame by adopting a track segment association method; b. respectively calculating the movement distance of a target pedestrian in the video frame and the movement distance of the pedestrians around the target pedestrian in the video frame in the whole tracking process; c. and judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian and the pedestrians around the target pedestrian in the video frame in the whole tracking process.
Wherein, the step a specifically comprises: generating a track segment of a pedestrian according to the video frame; and associating the generated pedestrian track segments by adopting the social relation distribution SAM characteristics to realize the tracking of the target pedestrian.
The surrounding pedestrians refer to: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
The step b of calculating the moving distance s of the target pedestrian in the video frame in the whole tracking process specifically comprises the following steps: every N frames, using the formula
Calculating the primary target pedestrian movement distance, wherein x and y are position coordinates of the target pedestrian, and L is the movement distance of the target pedestrian in the N frames; the moving distance s of the target pedestrian in the whole tracking process is as follows: l ═ S1+L2+…+Ln。
The step c specifically comprises the following steps: calculating the movement distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process through a beta function; calculating the difference value between the moving distance s of the target pedestrian and the calculated value of the beta function; if the difference is larger than a preset threshold value T, judging that abnormal behaviors occur; if the difference is smaller than a preset threshold value T, judging that no abnormal behavior occurs.
The invention also provides a pedestrian abnormal behavior detection system, which comprises a tracking module, a calculation module and a judgment module, wherein: the tracking module is used for tracking the target pedestrian in the video frame by adopting a track segment association method; the calculation module is used for calculating the movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process respectively; and the judging module is used for judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process.
Wherein the tracking module is specifically configured to: generating a track segment of a pedestrian according to the video frame; and associating the generated pedestrian track segments by adopting the social relation distribution SAM characteristics to realize the tracking of the target pedestrian.
The surrounding pedestrians refer to: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
The calculation module specifically calculates the movement distance s of the target pedestrian in the video frame in the whole tracking process as follows: every N frames, using the formula
Calculating the primary target pedestrian movement distance, wherein x and y are position coordinates of the target pedestrian, and L is the movement distance of the target pedestrian in the N frames; the moving distance s of the target pedestrian in the whole tracking process is as follows: l ═ S1+L2+…+Ln。
The judgment module is specifically configured to: calculating the movement distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process through a beta function; calculating the difference value between the moving distance s of the target pedestrian and the calculated value of the beta function; if the difference is larger than a preset threshold value T, judging that abnormal behaviors occur; if the difference is smaller than a preset threshold value T, judging that no abnormal behavior occurs.
According to the method and the system for detecting the abnormal behaviors of the pedestrians, disclosed by the invention, based on the tracking of the target pedestrians, the abnormal behaviors are detected not by a model but by comparing the movement inconsistency of the pedestrians and the surrounding pedestrians, so that the process of complex model learning is avoided. The pedestrian safety detection method and the pedestrian safety detection system can detect the wandering or lingering behavior of the pedestrian in the walking process, improve the efficiency of searching for the reason causing the safety problem by monitoring personnel, and save manpower.
Drawings
FIG. 1 is a flow chart of a method for detecting abnormal behavior of a pedestrian according to the present invention;
fig. 2 is a hardware architecture diagram of a pedestrian abnormal behavior detection system according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Fig. 1 is a flowchart illustrating an operation of a method for detecting abnormal pedestrian behavior according to a preferred embodiment of the present invention.
And step S1, tracking the target pedestrian in the video frame by adopting a track segment association method. Specifically, the method comprises the following steps:
firstly, generating a track segment of a pedestrian according to the video frame:
pedestrian detection is performed in the video frame by the HOG (Histogram of Oriented Gradient) method, and then the pedestrian is tracked by the optical flow method. Due to the existence of occlusion, the optical flow tracking is easily stopped, and a track segment of the pedestrian is generated.
Secondly, associating the generated pedestrian track segment by using a social relationship distribution (SAM) (social affinity map) characteristic to realize the tracking of the target pedestrian:
wherein, the social relationship (social affinity) is: the social relationship, namely the movement relationship of pedestrians around, can be composed of friends, relatives and working partners, such as the phenomenon of connector walking and connector-follower.
Firstly, vectorizing the generated pedestrian track segment to obtain the SAM feature of the track segment. Wherein the generated track segments of the pedestrians comprise track segments of the pedestrians to be tracked, namely target pedestrians. And then, clustering the track segments in a certain range around the track segment of the target pedestrian at the same moment according to the SAM characteristics by a clustering method. Where the range size is typically set to 3 meters, so that some outliers can be avoided. It should be noted that the moving direction and time of the track segment within a certain range around the track segment of the target pedestrian should coincide with the track segment.
Then, the result of the above clustering is described by a radial histogram, which is divided into ten regions, i.e. ten categories, according to the most common category of SAM features, and the radial histogram shows spatial position distribution of the ten categories.
And then, carrying out binary vectorization on the radial histogram to obtain a vector of the SAM characteristics.
And finally, associating the track segments through a Markov chain model (Markov-chain model), and comparing vectors of SAM (sample access model) characteristics of the two track segments through Hamming distance (Hamming distance) when the track segments are associated so as to associate the two track segments moving in similar social relationship distribution to form a long track of the target and finally realize the tracking of the target pedestrian.
And step S2, respectively calculating the moving distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process.
It is noted that the present embodiment is based on social relationships, so that in the association of trajectory segments, the trajectories of pedestrians within three meters around the target pedestrian are involved.
The pedestrians around the target pedestrian in the embodiment refer to: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
The following description will be given taking a specific calculation of the target pedestrian movement distance as an example:
the calculated target pedestrian movement distance refers to the movement distance of the target pedestrian in the whole tracking process.
Every N frames are spaced, the moving distance of the target pedestrian is calculated, and the formula is as follows:
The moving distance s of the target pedestrian in the whole tracking process is as follows:
S＝L1+L2+…+Ln。
the calculation method of the pedestrian movement distance around the target pedestrian is similar to the calculation method of the pedestrian movement distance of the target pedestrian, and the detailed description is omitted here.
And step S3, judging whether abnormal behaviors occur according to the calculated movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process. Specifically, the method comprises the following steps:
the moving distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process is calculated through a beta function, and the beta function can also be a variance calculation function or an average calculation function. And calculating a difference value between the movement distance s of the target pedestrian and the calculated value of the beta function, judging that abnormal behaviors occur if the difference value is larger than a preset threshold value T, and judging that no abnormal behaviors occur if the difference value is smaller than the preset threshold value T.
Fig. 2 is a hardware architecture diagram of the pedestrian abnormal behavior detection system according to the present invention. The system comprises a tracking module, a calculating module and a judging module.
The tracking module is used for tracking the target pedestrian in the video frame by adopting a track segment association method. Specifically, the method comprises the following steps:
firstly, generating a track segment of a pedestrian according to the video frame:
pedestrian detection is performed in the video frame by the HOG (Histogram of Oriented Gradient) method, and then the pedestrian is tracked by the optical flow method. Due to the existence of occlusion, the optical flow tracking is easily stopped, and a track segment of the pedestrian is generated.
Secondly, associating the generated pedestrian track segment by using a social relationship distribution (SAM) (social affinity map) characteristic to realize the tracking of the target pedestrian:
wherein, the social relationship (social affinity) is: the social relationship, namely the movement relationship of pedestrians around, can be composed of friends, relatives and working partners, such as the phenomenon of connector walking and connector-follower.
Firstly, vectorizing the generated pedestrian track segment to obtain the SAM feature of the track segment. Wherein the generated track segments of the pedestrians comprise track segments of the pedestrians to be tracked, namely target pedestrians. And then, clustering the track segments in a certain range around the track segment of the target pedestrian at the same moment according to the SAM characteristics by a clustering method. Where the range size is typically set to 3 meters, so that some outliers can be avoided. It should be noted that the moving direction and time of the track segment within a certain range around the track segment of the target pedestrian should coincide with the track segment.
Then, the result of the above clustering is described by a radial histogram, which is divided into ten regions, i.e. ten categories, according to the most common category of SAM features, and the radial histogram shows spatial position distribution of the ten categories.
And then, carrying out binary vectorization on the radial histogram to obtain a vector of the SAM characteristics.
And finally, associating the track segments through a Markov chain model (Markov-chain model), and comparing vectors of SAM (sample access model) characteristics of the two track segments through Hamming distance (Hamming distance) when the track segments are associated so as to associate the two track segments moving in similar social relationship distribution to form a long track of the target and finally realize the tracking of the target pedestrian.
The calculation module is used for calculating the movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process respectively.
It is noted that the present embodiment is based on social relationships, so that in the association of trajectory segments, the trajectories of pedestrians within three meters around the target pedestrian are involved.
The pedestrians around the target pedestrian in the embodiment refer to: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
The following description will be given taking a specific calculation of the target pedestrian movement distance as an example:
the calculated target pedestrian movement distance refers to the movement distance of the target pedestrian in the whole tracking process.
Every N frames are spaced, the moving distance of the target pedestrian is calculated, and the formula is as follows:
The moving distance s of the target pedestrian in the whole tracking process is as follows:
S＝L1+L2+…+Ln。
the calculation method of the pedestrian movement distance around the target pedestrian is similar to the calculation method of the pedestrian movement distance of the target pedestrian, and the detailed description is omitted here.
And the judging module is used for judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process. Specifically, the method comprises the following steps:
the moving distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process is calculated through a beta function, and the beta function can also be a variance calculation function or an average calculation function. And calculating a difference value between the movement distance s of the target pedestrian and the calculated value of the beta function, judging that abnormal behaviors occur if the difference value is larger than a preset threshold value T, and judging that no abnormal behaviors occur if the difference value is smaller than the preset threshold value T.
Although the present invention has been described with reference to the presently preferred embodiments, it will be understood by those skilled in the art that the foregoing description is illustrative only and is not intended to limit the scope of the invention, as claimed.
Claims (6)
1. A pedestrian abnormal behavior detection method is characterized by comprising the following steps:
a. tracking a target pedestrian in a video frame by adopting a track segment association method;
b. respectively calculating the movement distance of a target pedestrian in the video frame and the movement distance of the pedestrians around the target pedestrian in the video frame in the whole tracking process;
c. judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process;
the step a specifically comprises the following steps:
generating a track segment of a pedestrian according to the video frame;
adopting social relation distribution SAM characteristics to correlate the track segments of the generated pedestrians and realize the tracking of the target pedestrians; the social relationship distribution SAM characteristics are the motion relationship and social relationship of surrounding pedestrians, and are obtained by vectorizing track segments of target pedestrians;
the step c specifically comprises the following steps:
calculating the movement distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process through a beta function, wherein the beta function is a variance calculation function or an average calculation function;
calculating the difference value between the moving distance s of the target pedestrian and the calculated value of the beta function;
if the difference is larger than a preset threshold value T, judging that abnormal behaviors occur; and if the difference is smaller than a preset threshold value T, judging that no abnormal behavior occurs.
2. The method of claim 1, wherein said surrounding pedestrian is: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
3. The method as claimed in claim 2, wherein the step b of calculating the moving distance s of the target pedestrian in the video frame in the whole tracking process specifically comprises:
calculating the moving distance L of the target pedestrian once every N framesiI takes the value from 1 to n;
the moving distance s of the target pedestrian in the whole tracking process is as follows: l ═ S1+L2+…+Ln。
4. The system for detecting the abnormal behavior of the pedestrian is characterized by comprising a tracking module, a calculating module and a judging module, wherein:
the tracking module is used for tracking the target pedestrian in the video frame by adopting a track segment association method;
the calculation module is used for calculating the movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process respectively;
the judging module is used for judging whether abnormal behaviors occur or not according to the calculated movement distance of the target pedestrian in the video frame and the pedestrians around the target pedestrian in the video frame in the whole tracking process;
the tracking module is specifically configured to:
generating a track segment of a pedestrian according to the video frame;
adopting social relation distribution SAM characteristics to correlate the track segments of the generated pedestrians and realize the tracking of the target pedestrians; the social relationship distribution SAM characteristics are the motion relationship and social relationship of surrounding pedestrians, and are obtained by vectorizing track segments of target pedestrians;
the judgment module is specifically configured to:
calculating the movement distance of the target pedestrian and the pedestrians around the target pedestrian in the whole tracking process through a beta function, wherein the beta function is a variance calculation function or an average calculation function;
calculating the difference value between the moving distance s of the target pedestrian and the calculated value of the beta function;
if the difference is larger than a preset threshold value T, judging that abnormal behaviors occur; if the difference is smaller than a preset threshold value T, judging that no abnormal behavior occurs.
5. The system of claim 4, wherein said surrounding pedestrians are: pedestrians existing within three meters around the target pedestrian at the time of starting the tracking of the target pedestrian, and the final destination of the movement of these pedestrians is the same as that of the target pedestrian.
6. The system according to claim 5, wherein the calculating module calculates the moving distance s of the target pedestrian in the video frame in the whole tracking process specifically as follows:
calculating the moving distance L of the target pedestrian once every N framesiI takes the value from 1 to n;
the moving distance s of the target pedestrian in the whole tracking process is as follows: l ═ S1+L2+…+Ln。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510160836.2A CN106156705B (en) | 2015-04-07 | 2015-04-07 | Pedestrian abnormal behavior detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510160836.2A CN106156705B (en) | 2015-04-07 | 2015-04-07 | Pedestrian abnormal behavior detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106156705A CN106156705A (en) | 2016-11-23 |
| CN106156705B true CN106156705B (en) | 2020-08-25 |
Family
ID=57337677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510160836.2A Active CN106156705B (en) | 2015-04-07 | 2015-04-07 | Pedestrian abnormal behavior detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106156705B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711344B (en) * | 2018-12-27 | 2023-05-26 | 东北大学 | Front-end intelligent specific abnormal behavior detection method |
| CN111815496A (en) * | 2020-06-11 | 2020-10-23 | 浙江大华技术股份有限公司 | Association detection method and related equipment and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4623135B2 (en) * | 2008-05-08 | 2011-02-02 | 株式会社デンソー | Image recognition device |
| CN102289817B (en) * | 2011-07-19 | 2013-04-17 | 中国科学院自动化研究所 | Pedestrian counting method based on group context |
| CN102799863B (en) * | 2012-07-02 | 2014-12-17 | 中国计量学院 | Method for detecting group crowd abnormal behaviors in video monitoring |
| CN103839049B (en) * | 2014-02-26 | 2017-04-26 | 中国计量学院 | Double-person interactive behavior recognizing and active role determining method |
| CN103914854B (en) * | 2014-03-24 | 2016-08-17 | 河海大学 | A kind of image sequence target association and orbit generation method |
| CN104200488A (en) * | 2014-08-04 | 2014-12-10 | 合肥工业大学 | Multi-target tracking method based on graph representation and matching |
| CN104392445B (en) * | 2014-11-21 | 2017-10-31 | 中国计量学院 | Crowd's microcommunity dividing method in a kind of monitor video |
-
2015
- 2015-04-07 CN CN201510160836.2A patent/CN106156705B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| Abnormal Behavior Recognition Based on Trajectory Feature and Regional Optical Flow;A-Lin Hou et al.;《2013 Seventh International Conference on Image and Graphics》;20131024;全文 * |
| 视频中的大规模人群密度与异常行为分析;林沁;《中国优秀硕士学位论文全文数据库》;20140915;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106156705A (en) | 2016-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tian et al. | Online multi-object tracking using joint domain information in traffic scenarios | |
| Li et al. | DP-SLAM: A visual SLAM with moving probability towards dynamic environments | |
| Mneymneh et al. | Vision-based framework for intelligent monitoring of hardhat wearing on construction sites | |
| US10217229B2 (en) | Method and system for tracking moving objects based on optical flow method | |
| CN106156706B (en) | Pedestrian abnormal behavior detection method | |
| US8218819B2 (en) | Foreground object detection in a video surveillance system | |
| US9530221B2 (en) | Context aware moving object detection | |
| US20110142283A1 (en) | Apparatus and method for moving object detection | |
| US9305217B2 (en) | Object tracking system using robot and object tracking method using a robot | |
| CN107170011B (en) | robot vision tracking method and system | |
| US9922423B2 (en) | Image angle variation detection device, image angle variation detection method and image angle variation detection program | |
| US9904868B2 (en) | Visual attention detector and visual attention detection method | |
| Bhaskar et al. | Autonomous detection and tracking under illumination changes, occlusions and moving camera | |
| CN104408406A (en) | Staff off-post detection method based on frame difference method and background subtraction method | |
| Tsintotas et al. | DOSeqSLAM: Dynamic on-line sequence based loop closure detection algorithm for SLAM | |
| Sun et al. | Region of interest constrained negative obstacle detection and tracking with a stereo camera | |
| CN106156705B (en) | Pedestrian abnormal behavior detection method and system | |
| CN103646242A (en) | Maximally stable extremal region characteristic-based extended target tracking method | |
| CN110688969A (en) | Video frame human behavior identification method | |
| Chen et al. | Lane detection by trajectory clustering in urban environments | |
| CN113971697A (en) | Air-ground cooperative vehicle positioning and orienting method | |
| CN102789645A (en) | Multi-objective fast tracking method for perimeter precaution | |
| Uke et al. | Motion tracking system in video based on extensive feature set | |
| CN115909219A (en) | Scene change detection method and system based on video analysis | |
| CN104376579A (en) | Moving object detecting method and system used for online class system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |