WO2021114766A1

WO2021114766A1 - Method and system for analyzing behavior pattern of person on the basis of depth data

Info

Publication number: WO2021114766A1
Application number: PCT/CN2020/113952
Authority: WO
Inventors: 邵肖伟; 许永伟
Original assignee: 深圳市鸿逸达科技有限公司
Priority date: 2019-12-09
Filing date: 2020-09-08
Publication date: 2021-06-17
Also published as: CN111723633A; CN111723633B

Abstract

A method for analyzing a behavior pattern of a person on the basis of depth data, comprising: deploying a plurality of distance sensors, and collecting depth data in a detection region (S1); processing the collected depth data, and forming structured data (S2); and analyzing a behavior pattern of a target, determining whether or not the target has displayed abnormal behavior, and issuing an alert if so (S3). The method can achieve the following objectives: 1) automatic detection: monitoring the current posture of a person in real time to detect abnormal behavior, and automatically issuing an alert upon detection of abnormal behavior; and (2) behavior analysis: continuously monitoring behavior of a person to analyze lifestyle and habits of said person.

Description

A method and system for analyzing personnel behavior patterns based on in-depth data

Technical field

The present invention relates to pedestrian detection and point cloud intelligent analysis technology, and more specifically, to a method and system for analyzing personnel behavior patterns based on depth data.

Background technique

For video surveillance of people in indoor environments, conventional color cameras are generally used to manually monitor people's behavior, and some use machine vision technology to automatically monitor people's behavior based on color video. The use of video to monitor personnel and monitor abnormal behavior plays an important role in the field of security. The current questions are: 1. In some judicial security fields, it is not suitable to use conventional color cameras. How to protect the privacy of the monitored persons. 2. The use of machine vision technology based on color video can monitor certain personnel behavior, but cannot locate the spatial information of the monitoring personnel. 3. Night monitoring, conventional color cameras cannot monitor personnel behavior. 4. Alarm of abnormal video surveillance.

Summary of the invention

In order to solve the problems in the prior art, the present invention proposes a method for analyzing personnel behavior patterns based on depth data, which includes: S1, deploying multiple distance sensors to collect depth data in the detection area; S2, performing analysis on the collected depth data Process to form structured data; S3, analyze the target's behavior pattern, determine the target's abnormal behavior, and if it is abnormal, give an alarm.

The present invention also proposes a personnel behavior pattern analysis system based on depth data, which includes a memory storing a computer program and a processor capable of executing the computer program, and the computer program is executed to implement the following steps: S1, deploy multiple units The distance sensor collects depth data in the detection area; S2, processes the collected depth data to form structured data; S3, analyzes the behavior pattern of the target, judges the abnormal behavior of the target, and if it is abnormal, alarms.

The invention adopts the personnel behavior pattern analysis technology based on in-depth data to monitor personnel. Install distance sensors indoors, collect personnel's depth data, use intelligent algorithms to automatically monitor personnel behavior, and automatically alert abnormal behaviors. The following problems have been solved: 1. Automatic detection: real-time monitoring of the current personnel posture and detection of abnormal behaviors. When abnormal behavior occurs, it will automatically alert. 2. Behavior analysis: Continuously monitor the behavior of personnel and analyze the living habits and habits of personnel.

The beneficial effects of the present invention include: 1. High-precision spatial data collection: millimeter-level recognition accuracy. 2. Strong data resolving ability: accurate object recognition, accurate identification of behaviors and actions. 3. No external light is required. It is not affected by the external environment such as night and fog. 4. Privacy protection: Does not recognize facial information. 5. Active intelligent judgment and early warning: structured data, rule-based data detection and active early warning. 6. Intelligent linkage: Drive traditional security equipment such as video surveillance, sound and light alarms based on the results of rule judgments, and coordinate linkage.

Description of the drawings

Fig. 1 is a flowchart of an embodiment of the method of the present invention.

Fig. 2 is a flowchart of an embodiment of the method of the present invention.

Detailed ways

The embodiments of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the method of one embodiment of the present invention is as follows.

S1, deploy multiple distance sensors to collect depth data in the detection area.

The distance sensor is arranged in the air. The sensor upgrades the existing application level from two-dimensional to three-dimensional, presents management objects from a spatial perspective, improves security management capabilities with structured spatial data, and realizes active security supported by accurate data.

Each distance sensor is a client, which communicates through wireless LSN, and the receipt collected by the sensor is returned to the host server. The host server can view the working status of each sensor, and use the data assimilation algorithm to collect and integrate the data of all sensors to form a God's perspective and view the flow of people in the entire area in real time.

The distance sensor emits and receives invisible light beams through the area scanning method, and obtains the scanning data of each frame. The data includes the distance from the scanned object to the sensor, scanning time and scanning frequency.

Preferably, before data collection, the distance sensor needs to be processed as follows:

1. Synchronization: Before data collection starts, each client needs to synchronize time. Even if the devices are time synchronized before the acquisition, the acquisition PC and sensor will still have a slight time error during long-term work. These errors need to be manually eliminated in the later stage to make the data time completely consistent.

2. Registration: Place multiple sensor data point clouds in a world coordinate display to form a God's perspective; select a plane data point in the original point cloud data space, such as a ground data point, and convert the ground data point to the same plane data point , The least square method is used to minimize the distance between the ground data point and this plane, so as to obtain a transformation parameter, and then calculate the relative coordinates of each sensor.

S2, processing the collected depth data to form structured data. The step S2 includes:

S21, the depth data is preprocessed, including: 1) Data denoising: Due to the influence of hardware equipment and the actual environment, the depth data collected by the depth sensor has some noise. According to the neighborhood information of each data point, a certain size of filter is set The operator uses a two-dimensional convolution operation to remove invalid data points, which can improve the accuracy of target detection; 2) Data normalization: Due to the influence of the actual environment, there are some invalid distance data in the depth data collected by the depth sensor. By selecting an effective detection distance, the distance value is converted to a linear transformation between [0-1]. This normalized depth data can eliminate the influence of different target distances on the detection, which is beneficial to mention the target inspection accuracy; (

S22: For the preprocessed depth data, according to information such as collection time, data size, depth data, and point cloud data, create a depth data database as needed for tasks such as target detection, real-time warning, and behavior analysis.

S3: Analyze the behavior pattern of the target, determine the abnormal behavior of the target, and give an alarm if it is abnormal. Specifically, the depth data is compared with the determined background model library, and the behavior pattern information of the target is obtained, thereby determining that the target belongs to a sitting posture, a standing posture, or a lying posture.

When analyzing the behavior pattern of the target, the following steps are specifically included:

S31. The behavior mode of collecting and marking the target. Perform deep learning detection and analysis on the current depth data, manually label the acquired structured data according to the target task, and label the category as sitting, standing or lying position, and the label information is the boundary information of the target in the depth image.

S32. Train a classifier for target location and category detection through a deep neural network. Preprocess the current depth data, use the trained target detection model, and get the target detection result of the current frame by forward calculation, and detect whether the target is sitting, standing or lying posture information; combined with the point cloud space information of the corresponding target Get the coordinate position of the target; finally get the spatial position information and behavior pattern category of the target.

The deep neural network can adopt the SSD network structure (see Liu W, Snguelov D, ErhSn D, et Sl. SSD: Single Shot MultiBox Detector), and continuously train the network parameters through the network output results and the gaps in the actual results. The purpose of the present invention is to detect the location and category information of a person, and the deep learning SSD network structure combines convolution operation and regression analysis to obtain the location and category of the target. The SSD network structure uses convolution operation to obtain the feature information of the target, regression analysis to obtain the location and category of the target, and then combines the multi-scale feature information to precisely determine the target location.

S33: Use the obtained behavior pattern classifier to predict the behavior category of the target in the depth data, judge the abnormal behavior according to the detected category result, and send an alarm when the abnormal behavior is detected. After the spatial coordinate position and posture category of the target are obtained, the target position and posture of the person in the detection area can be determined, and then the personnel's behavior can be judged through the temporal spatial position and posture information of the person. Finally, the behavior is detected, and when irregularities or violations occur, the alarm mechanism is triggered, and real-time alarms are triggered. For example, in a monitored room, when there is a person's super-elevation self-hanging behavior, the background detection software will detect that there is a person's continuous height abnormality at a certain position, so as to trigger the super-elevation self-hanging alarm in real time; such as in a monitored room If there is a person who does not take a break within the specified time, the background detection software will detect that someone has not lie down and rest at the specified time, thereby triggering an alarm of irregular behavior in real time.

S4, analyze the target behavior.

Fusion of multiple distance sensor detection results, based on long-term behavior data to analyze the target life habits. After obtaining the target's spatial position and posture category, the status of the target person can be monitored in real time, and the status information of the target over a long period of time can also be obtained. Analysis of the status information of the target over a long period of time includes:

1) Motion trajectory analysis: According to the position and posture of the target person, obtain the motion and action trajectory of the target person;

2) Analysis of work and rest habits: According to the target person's position and posture, get the target person's starting and ending sleep time, sleep duration and sleep quality, etc., so as to get the target person's work and rest habits;

3) Communication habit analysis: According to the location and posture of the target person, obtain information such as the frequency of communication between the target person and other persons, communication time, and individual status, so as to obtain the communication habits of the target person. Psychological analysis and psychological treatment of target personnel can be performed;

The present invention can intelligently recognize the posture of indoor personnel, and indoor monitoring can be applied to the following scenarios: 1. Indoor monitoring and early warning of schools, kindergartens, judicial institutions, and government agencies: real-time real-time monitoring of crowd gathering, abnormal crowd behavior and abnormal individual behavior, etc. Detection and early warning. 2. Early warning of mass incidents and abnormal events in traffic hubs and public places: real-time grasp of passenger distribution, regional density, and speed of action, plan scientific paths for emergency relief, cross-border warnings, illegal intrusion warnings, and cluster warnings linked to the broadcasting system .

The above are only preferred specific embodiments of the present invention. It should be pointed out that the usual changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

A method for analyzing personnel behavior patterns based on in-depth data, which is characterized in that it includes:

S1, deploy multiple distance sensors to collect depth data in the detection area;

S2, processing the collected depth data to form structured data;

S3: Analyze the behavior pattern of the target, determine the abnormal behavior of the target, and give an alarm if it is abnormal.
The method according to claim 1, characterized in that, in step S1, further comprising:

1) Before the start of data collection, time synchronize each distance sensor;

2) Configure each distance sensor and calculate the relative coordinates of each sensor.
The method according to claim 1, characterized in that, in step S2, further comprising:

S21, preprocessing the depth data, including data denoising and data normalization;

S22: For the preprocessed depth data, according to the collection time, data size, depth data, and point cloud data information, create a depth data database as needed.
The method according to claim 1, characterized in that, in step S3, further comprising:

S31. The behavior mode of collecting and labeling the target: performing deep learning detection and analysis on the current deep data, and manually labeling the obtained structured data according to the target task;

S32. Train a classifier for target location and category detection through a deep neural network.

S33: Use the obtained behavior pattern classifier to predict the behavior category of the target in the depth data, judge the abnormal behavior according to the detected category result, and send an alarm when the abnormal behavior is detected.
The method according to claim 4, characterized in that, in step S32, further comprising:

1) Preprocess the current depth data, use the trained target detection model, and obtain the target detection result of the current frame by forward calculation, and detect whether the target is sitting, standing or lying posture information;

2) Combining the point cloud space information of the corresponding target to obtain the coordinate position of the target;

3) Obtain the target's spatial location information and behavior pattern category.
The method according to claim 5, characterized in that, in step S33, further comprising:

The SSD network structure is used for training, and the network parameters are continuously trained through the gap between the network output results and the actual results.
The method according to claim 4, characterized in that, in step S33, further comprising:

After the spatial coordinate position and posture category of the target are obtained, the target position and posture of the person in the detection area can be determined, and then the personnel's behavior can be judged through the temporal spatial position and posture information of the person.
A human behavior pattern analysis system based on depth data, characterized in that it comprises a memory storing a computer program and a processor capable of executing the computer program, and the computer program implements the following steps when the computer program is executed:

S1, deploy multiple distance sensors to collect depth data in the detection area;

S2, processing the collected depth data to form structured data;

S3: Analyze the behavior pattern of the target, determine the abnormal behavior of the target, and give an alarm if it is abnormal.
The system according to claim 8, wherein:

Step S1 also includes: 1) Time synchronization of each distance sensor before the start of data collection; 2) Configure each distance sensor and calculate the relative coordinates of each sensor;

Step S2 also includes: S21, preprocessing the depth data, including data denoising and data normalization; S22, on the preprocessed depth data, according to the acquisition time, data size, depth data and point cloud data information, according to Need to make an in-depth data database;

Step S3 also includes: S31, the behavior mode of collecting and labeling the target: performing deep learning detection and analysis on the current deep data, and manually labeling the obtained structured data according to the target task; S32, training a target location and category through a deep neural network The detected classifier. S33: Use the obtained behavior pattern classifier to predict the behavior category of the target in the depth data, judge the abnormal behavior according to the detected category result, and send an alarm when the abnormal behavior is detected.
The system according to claim 9, wherein:

Step S32 also includes: 1) Preprocessing the current depth data, using the trained target detection model, forward calculation to obtain the target detection result of the current frame, and detecting whether the target is sitting, standing or lying posture information; 2 ) Combining the point cloud space information of the corresponding target to obtain the coordinate position of the target; 3) Obtain the target's spatial position information and behavior pattern category;

Step S33 also includes: using the SSD network structure for training, and continuously training network parameters through the gap between the network output results and the actual results.