WO2020221031A1

WO2020221031A1 - Behavior thermodynamic diagram generation and alarm method and apparatus, electronic device and storage medium

Info

Publication number: WO2020221031A1
Application number: PCT/CN2020/085423
Authority: WO
Inventors: 赵飞
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2019-04-28
Filing date: 2020-04-17
Publication date: 2020-11-05
Also published as: CN111862521B; CN111862521A

Abstract

A behavior thermodynamic diagram generation and alarm method and apparatus, an electronic device and a storage medium. Said method comprises: acquiring a behavior analysis result of each designated target in image data of each pre-set monitored region, the pre-set monitored region being a region in a region to be counted; according to the behavior analysis result of each designated target, determining the frequency of the designated behavior occurring in each sub-region of the region to be counted, there being an intersection between the sub-region and the pre-set monitored region; and according to the frequency of the designated behavior occurring in each sub-region, generating a behavior thermodynamic diagram of the designated behavior in the region to be counted. The behavior thermodynamic diagram generation method in the embodiments of the present application counts, by means of image data, the frequency of a designated behavior occurring in each sub-region of a region to be counted, and then generates a behavior thermodynamic diagram of the region to be counted, thereby realizing intuitive monitoring of a large-area region.

Description

Behavior heat map generation and alarm method, device, electronic equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 28, 2019 with the application number 201910351634.4 and the invention titled "Methods, devices, electronic equipment, and storage media for generating and alarming behavioral heat maps", and its entire contents Incorporated in this application by reference.

Technical field

This application relates to the field of image processing technology, in particular to methods, devices, electronic equipment, and storage media for generating and warning behavioral heat maps.

Background technique

With the development of computer vision technology, it has become possible to automatically identify designated targets through image data. In related image recognition technologies, computer vision technologies, such as convolutional neural networks, are used to analyze image data to determine whether a target in the image data has a specified behavior. However, the above method only recognizes whether a specified behavior occurs in the single-channel image data, and it is inconvenient to visually monitor a large area.

Summary of the invention

The purpose of the embodiments of the present application is to provide a method, device, electronic device, and storage medium for generating and alarming behavioral heat map to realize intuitive monitoring of a large area. The specific technical solutions are as follows:

In the first aspect, an embodiment of the present application provides an alarm method, and the method includes:

Displaying the behavior heat map of the area to be counted, wherein the behavior heat map represents the frequency of occurrence of a specified behavior in each sub-region of the area to be counted;

When the sub-area of the behavior heat map meets the preset alarm condition, an alarm for the sub-area meeting the preset alarm condition is triggered.

Optionally, when the sub-area in the behavioral heat map meets a preset alarm condition, triggering an alarm for the sub-area meeting the preset alarm condition includes:

Respectively comparing the frequency of occurrence of the specified behavior in each sub-region of the behavior heat map with the magnitude of the preset frequency threshold;

For the target sub-area whose frequency of occurrence of the specified behavior is greater than the preset frequency threshold, trigger an alarm for the target sub-area.

Optionally, the sub-regions of the behavior heat map include thermal colors, and the thermal colors represent the frequency of occurrence of a specified behavior in the sub-region, and the higher the frequency of occurrence of the specified behavior in the sub-region, the The higher the heating value of the thermal color of the area;

The triggering an alarm for the sub-area meeting the preset alarm condition when the sub-area of the behavior heat map meets the preset alarm condition includes:

Respectively comparing the thermal value of the thermal color of each of the sub-regions with the preset thermal pre-value;

For the sub-area to be alarmed with a heating value greater than the preset heating threshold, an alarm for the sub-area to be alarmed is triggered.

Optionally, the sub-regions of the behavior heat map include thermal colors, the designated behaviors are multiple designated behaviors, and different designated behaviors correspond to different thermal colors, and the depth of the thermal color corresponds to the thermal color. The frequency of occurrence of the specified behavior is positively correlated, and each of the thermal colors corresponds to the corresponding alarm linkage;

For each thermal color in each of the sub-regions, comparing the depth of the thermal color with the size of the preset degree preset value corresponding to the thermal color;

For the target thermal color whose depth is greater than the preset degree preset value, trigger an alarm linkage for the sub-region where the target thermal color is located and corresponding to the target thermal color.

Optionally, the alarm method in the embodiment of the present application further includes:

Obtain the user's display instructions for the sub-areas to be displayed;

According to the display instruction, display the image data in the sub-region to be displayed, wherein the image data in the sub-region to be displayed is a video stream of the monitoring area in the sub-region to be displayed.

In the second aspect, an embodiment of the present application provides a method for generating a behavioral heat map, which is applied to a back-end device, and the method includes:

Acquiring the behavior analysis result of each designated target in the image data of each preset monitoring area, where the preset monitoring area is an area in the area to be counted;

Determine the frequency of occurrence of the specified behavior in each sub-region of the area to be counted according to the behavior analysis result of each designated target, wherein the sub-region and the preset monitoring area have an intersection;

According to the frequency of occurrence of the designated behavior in each of the sub-regions, a behavior heat map of the designated behavior in the region to be counted is generated.

Optionally, the obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area includes:

Obtain image data of each preset monitoring area;

Using computer vision technology to track and detect the designated targets in each of the image data, and extract the pixel area sequence of each designated target;

The pixel area sequence of each designated target is analyzed, and the behavior analysis result of each designated target is obtained.

Optionally, the pixel region sequence of each designated target is the pixel region sequence of each sampled designated target, and the computer vision technology is used to track and detect the designated target in each of the image data, and extract the A sequence of pixel areas, including:

Determine each designated target in each of the image data and the position of each designated target through a preset target detection algorithm and a preset target tracking algorithm;

Sampling each designated target in each of the image data by a preset target sampling algorithm to obtain each sampling designated target;

According to the position of the designated target of each sample, target behavior sequence extraction is performed on each of the image data to obtain the pixel area sequence of each designated target of the sample.

Optionally, one of the sub-regions includes at least one of the preset monitoring regions, and the determining the frequency of occurrence of the designated behavior in each sub-region of the region to be counted according to the behavior analysis result of each designated target includes:

Acquiring the inclusion relationship between each of the sub-regions and each of the preset monitoring regions;

According to the inclusion relationship, the behavior analysis result of each designated target, and the preset monitoring area where each designated target is located, the frequency of occurrence of the designated behavior in each of the sub-regions of the area to be counted is determined.

Optionally, after obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area, the method further includes:

Acquiring the actual position of each designated target in the preset monitoring area;

The determining the frequency of occurrence of the designated behavior in each sub-region of the area to be counted according to the behavior analysis result of each designated target includes:

According to the actual location of each designated target and the behavior analysis result of each designated target, the frequency of occurrence of the designated behavior in each sub-region of the area to be counted is determined.

Optionally, the acquiring the actual position of each of the designated targets in the preset monitoring area includes:

Determine the position of each designated target in the image data according to the pixel area sequence of each designated target;

According to the position of each designated target in the image data, the actual position of each designated target in the preset monitoring area is determined.

Optionally, the determining the frequency of occurrence of the specified behavior in each subregion of the area to be counted according to the actual location of each specified target and the behavior analysis result of each specified target includes:

According to the behavior analysis result of each designated target, classify each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

Determine the target behavior list corresponding to the specified behavior;

According to the actual position of each designated target in the target behavior list, the frequency of occurrence of the designated behavior in each sub-region of the area to be counted is determined.

Optionally, before the determining the frequency of occurrence of the designated behavior in each subregion of the area to be counted according to the behavior analysis result of each designated target, the method further includes:

Acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of the sub-region;

According to the granularity setting instruction, each sub-area in the area to be counted is determined.

Optionally, the designated behavior has a plurality of designated behaviors, and the generating a behavior heat map of the designated behavior of the region to be counted according to the frequency of occurrence of the designated behavior in each of the sub-regions includes:

Acquiring an electronic map of the area to be counted, and acquiring the frequency of occurrence of each designated behavior in each of the sub-regions;

Determine the thermal color corresponding to each specified behavior;

For any subregion in the electronic map, according to the frequency of occurrence of each designated behavior in the subregion, the thermal color corresponding to each designated behavior in the subregion is displayed on the map of the subregion, where any The intensity of the thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.

Optionally, the behavior analysis result of each designated target is a list of designated targets that trigger each designated behavior; the obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area includes:

Receiving each behavior list sent by each front-end smart device, where the behavior list includes the identification of the designated target, and the behavior types of the designated targets in the same behavior list are the same;

Assemble each of the behavior lists, and obtain the designated target lists that trigger each designated behavior.

In the third aspect, an embodiment of the present application provides a method for sending a behavior list, which is applied to a front-end smart device, and the method includes:

Obtain image data of the preset monitoring area;

Analyze the image data by computer vision technology to obtain the behavior analysis result of each designated target in the image data;

According to the behavior analysis result of each designated target, classify each designated target to obtain multiple behavior lists, wherein the behavior type of each designated target in the same behavior list is the same;

Send a list of each of the described actions.

Optionally, the behavior analysis result of each designated target is the behavior analysis result of each sampled designated target. The image data is analyzed by computer vision technology to obtain the behavior analysis result of each designated target in the image data, include:

Determine each designated target and the position of each designated target in the image data by using a preset target detection algorithm and a preset target tracking algorithm;

Sampling each designated target in the image data by using a preset target sampling algorithm to obtain each sampling designated target;

Performing target behavior sequence extraction on the image data according to the positions of the designated targets of each sample to obtain a sequence of pixel regions of the designated targets of each sample;

Analyze the pixel area sequence of each of the sampled designated targets to obtain the behavior analysis result of each of the sampled designated targets.

In a fourth aspect, an embodiment of the present application provides an alarm device, which includes:

The heat map display module is used to display the behavior heat map of the area to be counted, wherein the behavior heat map represents the frequency of occurrence of a specified behavior in each sub-region of the area to be counted;

The alarm triggering module is used to trigger an alarm for the sub-area meeting the preset alarm condition when the sub-area of the behavior heat map meets the preset alarm condition.

Optionally, the alarm trigger module includes:

A frequency comparison sub-module for comparing the frequency of occurrence of a specified behavior in each sub-region of the behavior heat map with a preset frequency threshold;

The sub-area alarm sub-module is used for triggering an alarm for the target sub-area for the target sub-area whose frequency of occurrence of the specified behavior is greater than the preset frequency threshold.

The alarm trigger module includes:

The thermal value comparison sub-module is used to compare the thermal value of the thermal color of each of the sub-regions with the preset thermal value;

The triggering alarm sub-module is used for triggering an alarm for the sub-area to be alarmed whose heating value is greater than the preset heating threshold.

The alarm trigger module is specifically used for:

Optionally, the alarm device in the embodiment of the present application further includes:

The display instruction receiving module is used to obtain the user's display instruction for the sub-area to be displayed;

The image data display module is configured to display the image data in the sub-region to be displayed according to the display instruction, wherein the image data in the sub-region to be displayed is the monitoring in the sub-region to be displayed The video stream of the region.

In a fifth aspect, an embodiment of the present application provides an apparatus for generating a behavioral heat map, which is applied to a back-end device, and the apparatus includes:

The analysis result obtaining module is used to obtain the behavior analysis result of each designated target in the image data of each preset monitoring area, wherein the preset monitoring area is the area in the area to be counted;

The sub-region frequency statistics module is used to determine the frequency of occurrence of the designated behavior in each sub-region of the region to be counted according to the behavior analysis result of each designated target, wherein the sub-region and the preset monitoring region exist Intersection

The behavior heat map generating module is configured to generate a behavior heat map of the designated behavior of the region to be counted according to the frequency of occurrence of the designated behavior in each of the sub-regions.

Optionally, the analysis result obtaining module includes:

Image data acquisition sub-module for acquiring image data of each preset monitoring area;

The behavior analysis sub-module, the behavior analysis sub-module includes:

An area sequence determination unit, configured to track and detect the designated targets in each of the image data using computer vision technology, and extract the pixel region sequence of each of the designated targets;

The area sequence analysis unit is used to analyze the pixel area sequence of each designated target to obtain the behavior analysis result of each designated target.

Optionally, the pixel region sequence of each designated target is a pixel region sequence of each sample designated target, and the region sequence determining unit includes:

The position determination subunit is used to determine each designated target in each of the image data and the position of each designated target through a preset target detection algorithm and a preset target tracking algorithm;

The coefficient sampling subunit is used to sample each designated target in each of the image data by using a preset target sampling algorithm to obtain each sampled designated target;

The region intercepting and determining subunit is used to perform target behavior sequence extraction on each of the image data according to the position of the designated target of each sample to obtain the pixel region sequence of each designated target of the sample.

Optionally, one of the sub-regions includes at least one of the preset monitoring regions, and the sub-region frequency statistics module includes:

An inclusion relationship determination sub-module for obtaining the inclusion relationship between each of the sub-regions and each of the preset monitoring regions;

The behavior frequency statistics sub-module is used to determine the occurrence of a designated behavior in each of the sub-regions of the area to be counted according to the inclusion relationship, the behavior analysis result of each of the designated targets, and the preset monitoring area where each designated target is The frequency.

Optionally, the device for generating a behavioral heat map in this embodiment of the present application further includes:

An actual position acquisition module, configured to acquire the actual position of each designated target in the preset monitoring area;

The sub-region frequency statistics module is specifically configured to determine the frequency of occurrence of the designated behavior in each sub-region of the region to be counted according to the actual location of each designated target and the behavior analysis result of each designated target.

Optionally, the actual location acquisition module includes:

An image position acquisition sub-module, configured to determine the position of each designated target in the image data according to the pixel area sequence of each designated target;

The actual position mapping sub-module is used to determine the actual position of each designated target in the preset monitoring area according to the position of each designated target in the image data.

Optionally, the sub-region frequency statistics module includes:

The designated target classification sub-module is used to classify each designated target according to the behavior analysis result of each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The target list determination sub-module is used to determine the target behavior list corresponding to the specified behavior;

The frequency determination sub-module is used to determine the frequency of occurrence of the specified behavior in each sub-region of the area to be counted according to the actual position of each designated target in the target behavior list.

A setting instruction acquisition module, configured to acquire a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of the sub-region;

The sub-region setting module is used to determine each sub-region in the area to be counted according to the granularity setting instruction.

Optionally, the specified behavior has multiple specified behaviors, and the behavior heat map generating module includes:

The multi-frequency statistics sub-module is used to obtain an electronic map of the area to be counted, and obtain the frequency of occurrence of each designated behavior in each sub-area;

The thermal color corresponding sub-module is used to determine the thermal color corresponding to each specified behavior;

The map coloring sub-module is used for any sub-area in the electronic map, according to the frequency of occurrence of each specified behavior in the sub-area, display the corresponding to each specified behavior in the sub-area on the map of the sub-area Thermal color, where the intensity of any thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.

Optionally, the behavior analysis result of each designated target is a list of designated targets that trigger each designated behavior; the analysis result obtaining module includes:

The behavior list receiving sub-module is configured to receive the behavior list sent by each smart device, wherein the behavior list includes the identification of the designated target, and the behavior types of the designated targets in the same behavior list are the same;

The behavior list assembling sub-module is used to assemble each of the behavior lists to obtain the designated target lists that trigger each designated behavior.

In a sixth aspect, an embodiment of the present application provides a behavior list sending device, which is applied to a front-end smart device, and the device includes:

The image data acquisition module is used to acquire the image data of the preset monitoring area;

The target behavior analysis module is used to analyze the image data through computer vision technology to obtain the behavior analysis result of each designated target in the image data;

The designated target classification module is configured to classify each designated target according to the behavior analysis result of each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The behavior list sending module is used to send each of the behavior lists.

Optionally, the behavior analysis result of each designated target is the behavior analysis result of each sampled designated target, and the target behavior analysis module includes:

The target position determination sub-module is used to determine each designated target in the image data and the position of each designated target through a preset target detection algorithm and a preset target tracking algorithm;

The designated target sampling sub-module is used to sample each designated target in the image data by using a preset target sampling algorithm to obtain each sampling designated target;

The pixel region interception sub-module is configured to extract the target behavior sequence of the image data according to the position of each of the sampled designated targets to obtain the pixel region sequence of each of the sampled designated targets;

The target behavior analysis sub-module is used to analyze the pixel area sequence of each of the sampling designated targets to obtain the behavior analysis result of each of the sampling designated targets.

In a seventh aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the alarm method described in any one of the first aspects when executing the program stored in the memory.

In an eighth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the method for generating a behavioral heat map according to any one of the second aspects when executing the program stored in the memory.

In a ninth aspect, an embodiment of the present application provides an electronic device including a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the behavior list sending method of any one of the foregoing third aspects when executing the program stored in the memory.

In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the foregoing aspects of the first aspect is implemented. 1. The alarm method described.

In an eleventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the above-mentioned second aspect when executed by a processor. Any of the described behavioral heat map generation methods.

In a twelfth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the foregoing third aspect is implemented Any of the aforementioned behavior list sending methods.

The behavioral heat map generation and alarm method, device, electronic equipment and storage medium provided in the embodiments of the application obtain the behavior analysis results of each designated target in the image data of each preset monitoring area, where the preset monitoring area is the area to be counted According to the behavior analysis results of each designated target, determine the frequency of the specified behavior in each sub-area of the area to be counted. Among them, the sub-area and the preset monitoring area overlap; according to the frequency of the specified behavior in each sub-area , To generate the behavior heat map of the specified behavior of the area to be counted. The frequency of occurrence of designated behaviors in each sub-region in the area to be counted is counted by the image data, and the behavior heat map of the area to be counted is generated, which can realize the intuitive monitoring of a large area. Of course, implementing any product or method of the present application does not necessarily need to achieve all the advantages described above at the same time.

Description of the drawings

In order to explain the embodiments of the present application and the technical solutions of the prior art more clearly, the following briefly introduces the drawings needed in the embodiments and the prior art. Obviously, the drawings in the following description are only the present For some of the embodiments of the application, for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative work.

FIG. 1a is a first schematic diagram of a method for generating a behavioral heat map according to an embodiment of this application;

FIG. 1b is a second schematic diagram of a method for generating a behavioral heat map according to an embodiment of the application;

FIG. 2 is a third schematic diagram of a method for generating a behavioral heat map according to an embodiment of the application;

FIG. 3 is a schematic diagram of an alarm method according to an embodiment of the application;

FIG. 4 is a schematic diagram of a deep learning algorithm training process according to an embodiment of the application;

FIG. 5 is a fourth schematic diagram of a method for generating a behavioral heat map according to an embodiment of the application;

FIG. 6 is a schematic diagram of a method for sending a behavior list according to an embodiment of the application;

FIG. 7 is a schematic diagram of an apparatus for generating a behavioral heat map according to an embodiment of the application;

FIG. 8 is a schematic diagram of an apparatus for sending a behavior list according to an embodiment of this application;

FIG. 9 is a schematic diagram of an electronic device according to an embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of this application.

In order to facilitate the user to visually monitor a large area, an embodiment of the present application provides a method for generating a behavioral heat map. See FIG. 1a, which is applied to a back-end device. The method includes:

S101: Obtain a behavior analysis result of each designated target in the image data of each preset monitoring area, where the foregoing preset monitoring area is an area in the area to be counted.

The method for generating a behavioral heat map in the embodiment of the present application is applied to a back-end device, so it can be executed by a back-end device. Specifically, the back-end device may be a server, a personal computer, or a hard disk video recorder. The image data in the embodiments of the present application may be a video stream, and in some application scenarios that only include target recognition, it may also be a single-frame video frame.

The preset monitoring area is the monitoring area designated by the user in the area to be counted. The back-end equipment can directly obtain the behavior analysis results of each designated target in each image data through smart cameras installed in each preset monitoring area. Among them, the smart camera analyzes the image data collected by itself through computer vision technology, obtains the behavior analysis result of each designated target in the image data collected by itself, and sends it to the back-end device.

Optionally, the smart camera can send the behavior analysis results of each specified target through a behavior list. For any smart camera, the smart camera establishes a behavior list for each behavior type, and the smart camera will trigger the designation of the specified behavior type The identification of the target is added to the corresponding behavior list. Correspondingly, obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area includes: receiving a behavior list sent by each smart device, wherein the behavior list includes the identification of the designated target, and the behavior list is in the same behavior list. The behavior types of the designated targets are the same; the above-mentioned behavior lists are assembled, and the designated target lists that trigger each designated behavior are respectively obtained, wherein the behavior analysis results of the above-mentioned designated targets are expressed in the form of the designated target list that triggers each designated behavior. By assembling the behavior list sent by each smart device, the back-end device can summarize the analysis results of the image data of multiple preset monitoring areas, and allocate the computing resources to the front-end smart device, reducing the processing pressure of the back-end device and improving flexibility. Specifically, the smart device here may be a smart camera or a hard disk video recorder, etc.

In a possible implementation manner, the image data analysis process is executed by a back-end device. See FIG. 1b. The above-mentioned acquiring behavior analysis results of each designated target in the image data of each preset monitoring area includes:

S1011: Obtain image data of each preset monitoring area.

The back-end device receives the image data of each preset monitoring area sent by each camera.

S1012: Analyze each of the above-mentioned image data through computer vision technology to obtain a behavior analysis result of the designated target in each of the above-mentioned image data.

The back-end equipment uses computer vision technology to obtain the behavior analysis results of the specified targets in each image data. The designated target is the target that the user wants to pay attention to, which can be a person, a vehicle, or an animal, etc., which can be set according to actual requirements. In a possible implementation manner, the computer vision technology is a pre-trained deep learning algorithm, and the back-end device uses the deep learning algorithm to analyze the image data to obtain the behavior analysis result of the specified target in the image data. The process of the pre-trained deep learning algorithm can be shown in Figure 4, including: determining the behavior type of interest, calibrating the behavior type of the specified target in each image data containing the specified target, obtaining sample image data, and inputting the sample image data Train in the deep learning algorithm, and get the pre-trained deep learning algorithm after convergence.

Optionally, the foregoing analysis of each of the foregoing image data by computer vision technology to obtain the behavior analysis result of the specified target in each of the foregoing image data includes:

Step 1: Using computer vision technology, the designated targets in each of the above-mentioned image data are respectively tracked and detected, and the pixel region sequence of each of the above-mentioned designated targets is extracted.

Computer vision technology can include target detection algorithms and target tracking algorithms. The back-end equipment recognizes the specified targets in each image data through the target detection algorithm, and tracks the specified targets through the target tracking algorithm to obtain each specified target According to the position in the image data, the pixel region sequence of each designated target is extracted according to the position of each designated target in the image data.

In a possible implementation manner, in order to reduce the processing pressure of the back-end device, the pixel area sequence of the designated target may be sampled. In the foregoing, the computer vision technology is used to track and detect the designated targets in each of the above-mentioned image data, and extract the pixel region sequence of each of the above-mentioned designated targets, including:

Step A: Determine each designated target and the position of each designated target in each of the aforementioned image data by using a preset target detection algorithm and a preset target tracking algorithm.

The back-end equipment recognizes the designated targets in each image data through target detection and calculation, and tracks the designated targets through the target tracking algorithm, so as to obtain the position of each designated target in the image data. In a possible implementation In the method, in order to effectively distinguish the designated targets, a unique ID can be set for each designated target. Target detection algorithms can include pedestrian target detection, such as HOG (Histogram of Oriented Gradient), DPM (Deformable Parts Models), FRCNN (Faster Regions with Convolutional Neural Networks, faster based on partitions) Convolutional Neural Network), YOLO (You Only Look Once), SSD (Single Shot Multibox Detector), the target tracking algorithm can be a multi-target tracking algorithm method.

Step B, sampling each designated target in each of the above-mentioned image data through a preset target sampling algorithm to obtain each sampling designated target.

The back-end equipment also performs target sampling on each designated target, thereby reducing the processing pressure of the back-end equipment. The back-end equipment can sample the specified target through any relevant sampling algorithm. For example, target sparse sampling of specified targets in each image data. Target sparse sampling methods include but are not limited to target uniform sampling, point uniform sampling, point weight sampling, and sampling based on the number of regional targets, etc., which can be obtained by sampling A designated target with an appropriate target scale, that is, sampling designated targets.

Step C: Perform target behavior sequence extraction on each of the above-mentioned image data according to the position of the designated target of each of the above-mentioned samples to obtain the pixel region sequence of each designated target of the above-mentioned sample.

In step A, the location of each designated target is determined, and each sampling designated target is a target in each designated target, so the position of each sampling designated target is known. The back-end device extracts the target behavior sequence of each image data according to the position of each sample designated target, and performs image interception from the image data according to a certain structure, such as Tubelet, to obtain the pixel area sequence of each sample designated target. By sampling the pixel area sequence of the designated target instead of the pixel area sequence of each designated target.

In the embodiment of this application, the pixel area sequence of each designated target is sampled, and the sparse sampling is completed according to the density of the target. While preserving the distribution characteristics of the designated target under different preset monitoring regions, it reduces the data of behavior type recognition processing. The quantity has improved the practicability of the overall scheme.

Step 2: Analyze the pixel area sequence of each of the above-mentioned designated targets to obtain the behavior analysis result of each of the above-mentioned designated targets.

Analyze the pixel region sequence of each specified target, using the image sequence behavior recognition framework, for example, LSTM (Long Short-Term Memory), dual stream network, C3D (3D ConvNets, deep 3-dimensional convolutional network) P3D (Pseudo-3D Residual Networks), ArtNet, PointNet, PointSIFT, etc., combine the classification neural network to extract the sequence behavior feature, and obtain the behavior analysis result of the pixel area sequence of each specified target. Among them, classification neural networks include, but are not limited to, Residual Neural Network 18 (Residual Neural Network 18), Residual Neural Network 50 (Residual Neural Network 50), Residual Neural Network 101 (Residual Neural Network 101), Resnet 152 (Residual Neural Network 152, Residual Neural Network 152), Inception-v1, VGG (Visual Geometry Group Network, Visual Geometry Group Network), etc. In a possible implementation manner, the behavior analysis result includes behavior category and confidence.

S102: Determine the frequency of occurrence of the designated behavior in each sub-region of the area to be counted according to the behavior analysis result of each of the above-mentioned designated targets, where the above-mentioned sub-region and the above-mentioned preset monitoring area have an intersection;

The area to be counted can be a preset area or a user-designated area. In a possible implementation manner, before S102, the above method further includes: acquiring a region to be counted selection instruction input by the user; and determining the region to be counted according to the above-mentioned region to be counted selection instruction. The selection command of the area to be counted represents the range of the area to be counted. Each sub-region of the area to be counted may be predetermined, for example, a plurality of area intervals are divided according to the area size in advance, and the size and division method of the sub-regions are set for each area interval. Determine the area interval to which the area to be counted belongs according to the size of the area to be counted, and determine each sub-area in the area to be counted according to the size and division method of the sub-areas corresponding to the area interval to which the area to be counted belongs. Of course, a fixed sub-region size and division method can also be set. For the region to be counted, each sub-region of the region to be counted is determined according to the fixed sub-region size and division method.

In a possible implementation manner, the sub-areas are divided in advance, and multiple sub-areas are divided in advance according to different granularities, for example, roads, floors, communities, urban areas, cities, or provinces. According to the pre-divided sub-areas, determine the sub-areas included in the area to be counted.

In a possible implementation manner, different granularities can also be selected according to user requirements. Optionally, before determining the frequency of occurrence of the designated behavior in each sub-region of the area to be counted based on the behavior analysis result of each designated target, the above method further includes:

Step 1: Obtain a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of the sub-region. The granularity setting instruction represents the size of the sub-region. For example, the granularity setting instruction represents the sub-region as a road, a floor, a district, an urban area, a city, or a province.

Step 2: Determine each sub-areas in the area to be counted according to the above-mentioned granularity setting instruction. For example, when the sub-area characterized by the granularity setting instruction is a cell, each sub-area is determined to be a cell; when the sub-area characterized by the granularity setting instruction is a street, it is determined that each sub-area is a street.

Through the granularity setting, the behavior analysis results of different granularities can be summarized, and the regional behavior can be visually displayed in different colors and color shades in combination with the electronic map, which is intuitive and easy to use.

The specified behavior can be a preset behavior type or a behavior type selected by the user in real time. In a possible implementation manner, the above method further includes: acquiring a specified behavior selection instruction input by a user, wherein the specified behavior selection instruction represents a behavior type of the specified behavior; and the specified behavior is determined according to the specified behavior selection instruction.

The back-end equipment determines the sub-region where each designated target belongs according to the preset monitoring area to which each designated target belongs; according to the behavior analysis results of each designated target, separately counts the frequency of occurrence of the designated behavior in each sub-region. Under normal circumstances, the size of the preset monitoring area is smaller than the size of the sub-area. In a possible implementation, the sub-area includes the preset monitoring area, and the determination is determined based on the behavior analysis result of each of the specified targets. The frequency of occurrence of the specified behavior in each sub-areas of the area to be counted, including:

Step 1: Obtain the inclusion relationship between each of the aforementioned sub-areas and each of the aforementioned preset monitoring areas.

According to the location of each sub-area and the location of each preset monitoring area, the preset monitoring areas included in each sub-area are respectively determined.

Step 2: Determine the frequency of occurrence of the specified behavior in each of the sub-regions of the area to be counted according to the inclusion relationship, the behavior analysis result of each of the specified targets, and the preset monitoring area where each of the specified targets is located.

The image data is a video image of the preset monitoring area, and the specified target in any image data is the specified target in the preset monitoring area corresponding to the image data. If the subarea includes a preset monitoring area, the designated target in the preset monitoring area is the designated target in the subarea. According to the behavior analysis results of each designated target, count the frequency of occurrence of designated behaviors in each sub-region.

In order to facilitate the statistics of the behavior analysis results of the designated targets, in a possible implementation manner, the above method further includes: classifying the designated targets according to the behavior analysis results of the designated targets to obtain multiple behavior lists, where , The behavior type of each specified target in the same behavior list is the same. According to the behavior analysis results of the specified target, each specified target of the same behavior type is divided into a behavior list. In addition to recording the corresponding behavior type and the identification of the specified target, the behavior list can also record the location of the specified target, and specify the target. The location of can be the image data/preset monitoring area to which the specified target belongs, or the location of the specified target is the actual coordinates of the specified target, etc.

S103: Generate a behavior heat map of the designated behavior in the region to be counted according to the frequency of occurrence of the designated behavior in each subregion.

The back-end device colors each sub-areas of the area to be counted in the electronic map according to the frequency of occurrence of the designated behavior in each sub-areas, so as to obtain a behavior heat map of the designated behavior in the area to be counted. In a possible implementation, cool and warm colors can be used to indicate the frequency of occurrence of the specified behavior in the sub-region. For example, the higher the frequency of the specified behavior in the sub-region, the closer the color of the sub-region to the warm color; The lower the frequency of the specified behavior, the closer the color of the sub-region is to the cool color.

In the actual application process, the user may wish to monitor multiple types of behaviors. In a possible implementation manner, the above-mentioned designated behaviors are multiple designated behaviors. The above-mentioned designated behaviors are generated according to the frequency of occurrence of the designated behaviors in each of the above-mentioned sub-regions. The behavior heat map of the specified behavior in the above-mentioned area to be counted includes:

Step 1: Obtain an electronic map of the area to be counted, and obtain the frequency of occurrence of each designated behavior in each of the sub-areas.

Step two, determine the thermal color corresponding to each of the above specified behaviors.

The designated behavior includes a variety of designated behaviors, and different thermal colors can be set for different designated behaviors. The thermal color corresponding to each specified behavior can be randomly determined or specified by the user, which will not be repeated here.

Step 3: For any sub-area in the above electronic map, according to the frequency of occurrence of each specified behavior in the sub-area, the thermal color corresponding to each specified behavior in the sub-area is displayed on the map of the sub-area, where any The intensity of a thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.

For any sub-region, the thermal color of the specified behavior contained in the sub-region is displayed in the electronic map position of the sub-region. And the higher the frequency of the specified behavior in the sub-region, the darker the thermal color corresponding to the specified behavior.

Use an electronic map as a base, use color shades to indicate the frequency of specified behaviors, use different colors to indicate different behavior types, and display the corresponding thermal colors in each sub-area position in the electronic map to obtain the behavior heat map of the area to be counted. In a possible implementation manner, the behavioral heat map can be zoomed in or out, and the frequency statistics data can be updated according to the scale of the electronic map. The user can select the image data in the sub-area from the electronic map for video preview to observe the actual situation more realistically. Optionally, the above method further includes obtaining an image display instruction for the designated preset monitoring area; and displaying the image data of the designated preset monitoring area according to the image display instruction. For example, the user can click the designated preset monitoring area in the sub-area through the mouse or touch screen, and the back-end device displays the image data of the designated preset monitoring area after detecting the click instruction for the designated preset monitoring area.

In the embodiment of the present application, the frequency of occurrence of designated behaviors in each sub-region in the region to be counted is counted by image data, and then a behavior heat map of the region to be counted is generated, which can realize intuitive monitoring of a large area.

When the granularity of the behavioral heat map is small, further positioning of the specified target is required. In a possible implementation manner, after acquiring the behavior analysis result of each designated target in the image data of each preset monitoring area, the above method further includes:

Acquire the actual position of each of the aforementioned designated targets in the aforementioned preset monitoring area.

The actual location of the designated target can be reported by a front-end smart device such as a smart camera, or it can be determined by a back-end device based on image data.

In a possible implementation manner, referring to FIG. 2, the foregoing obtaining the actual position of each of the specified targets in the preset monitoring area includes:

S201: Determine the position of each designated target in the image data according to the pixel area sequence of each designated target.

The pixel area of the specified target can be the pixel area selected by the target frame of the specified target. According to the pixel area sequence of the specified target, the position sequence of the specified target in the above image data is determined. For example, it can be a sequence of position coordinates (multiple consecutive time series). Coordinate areas).

S202: Determine the actual position of each designated target in the preset monitoring area according to the position of each designated target in the image data.

Through the relevant coordinate conversion method, the position of the designated target in the image data is converted into the actual position of the designated target in the preset monitoring area, and the actual position can be the global positioning system coordinates or a custom area coordinate.

In a possible implementation manner, the actual location of each designated target may be sent to the back-end device by a front-end device such as a smart camera, and the back-end device may directly obtain the actual location of each designated target.

The above determination of the frequency of occurrence of the designated behavior in each sub-region of the area to be counted based on the behavior analysis results of each of the above designated targets includes:

According to the actual location of each designated target and the behavior analysis result of each designated target, the frequency of occurrence of designated behaviors in each subregion of the area to be counted is determined.

Determine the actual location of each sub-areas of the area to be counted. According to the actual position of each designated target, the actual position of each sub-region, and the behavior analysis result of each designated target, the frequency of occurrence of the designated behavior in each sub-region is determined.

In the embodiment of the present application, by determining the actual location of each designated target, it can be applied to the case where the sub-region does not contain the complete preset monitoring area, and it can even be applied to the case where the sub-area is smaller than the preset monitoring area, which can be applied to behavior A scene with a small granularity of the heat map, that is, a scene with a small sub-area, theoretically the smallest sub-area can be a coordinate point, which can greatly increase the monitoring accuracy of the behavioral heat map.

In order to facilitate the statistics of the behavior analysis results of the designated targets, in a possible implementation manner, the designated behaviors in each subregion of the region to be counted are determined based on the actual position of each designated target and the behavior analysis result of each designated target. Frequency of occurrence, including:

S1021: According to the behavior analysis result of each of the specified targets, classify each of the specified targets to obtain multiple behavior lists, wherein the behavior types of the specified targets in the same behavior list are the same.

According to the behavior analysis result of the designated target, each designated target of the same behavior type is divided into a behavior list. For any behavior list, the behavior type of the behavior list, the identification of each specified target included in the behavior list, and the actual position of each specified target included in the behavior list are recorded in the behavior list.

S1022: Determine a target behavior list corresponding to the specified behavior.

Determine the behavior list corresponding to the specified behavior, that is, the target behavior list.

S1023: Determine the frequency of occurrence of the specified behavior in each subregion of the area to be counted according to the actual position of each designated target in the target behavior list.

Determine the actual location of each sub-areas of the area to be counted. According to the actual position of each designated target in the target behavior list and the actual position of each subregion, the frequency of occurrence of the designated target in each subregion, that is, the frequency of occurrence of the designated behavior in each subregion, is determined.

In the embodiment of the present application, by setting the behavior list, it is convenient to calculate the behavior analysis results of the specified target, and the behavior heat map generation efficiency is high.

The embodiment of the present application also provides an alarm method. Referring to FIG. 3, the method includes:

S301. Display the behavior heat map of the region to be counted, where the behavior heat map represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted.

The alarm method in the embodiment of the present application may be executed by a back-end device. Specifically, the back-end device may be a server, a personal computer, or a hard disk video recorder. The behavioral heatmap can be obtained by any of the above-mentioned methods for generating the behavioral heatmap, and will not be repeated here.

S302: When the sub-region of the above-mentioned behavioral heat map meets the preset alarm condition, trigger an alarm for the sub-region that meets the preset alarm condition.

The preset alarm condition can be set according to the actual situation, for example, the specified behavior frequency is greater than the preset frequency threshold, or the heat value of the heat color is greater than the preset heat preset value. In a possible implementation manner, when the sub-regions in the above-mentioned behavior heat map meet the preset alarm condition, triggering an alarm for the sub-regions that meet the preset alarm condition includes: comparing each sub-region of the above-mentioned behavior heat map respectively The frequency of occurrence of the specified behavior and the size of the preset frequency threshold; for the target subregion where the frequency of occurrence of the specified behavior is greater than the preset frequency threshold, an alarm for the target subregion is triggered.

In a possible implementation manner, the sub-regions of the above-mentioned behavior heat map include thermal colors, and the above-mentioned thermal colors represent the frequency of occurrence of the specified behavior in the above-mentioned sub-region, and the higher the frequency of occurrence of the specified behavior in the above-mentioned sub-region, the above-mentioned sub-region The higher the thermal value of the thermal color of the area; when the above-mentioned sub-regions of the above-mentioned behavioral heat map meet the preset alarm conditions, the alarm for the sub-regions meeting the preset alarm conditions is triggered, including: comparing the thermal colors of the above-mentioned sub-regions respectively The heating power value of and the preset heating power preset value; for the sub-area to be alarmed whose heating power value is greater than the preset heating threshold value, an alarm for the sub-area to be alarmed is triggered.

The method for generating a behavioral heat map of the embodiment of the present application may be specifically as shown in FIG. 5. The user can set the behavior category of interest, and the back-end device monitors the frequency of each specified behavior in the area to be counted in real time. Once the specified behavior category's thermal power increases significantly and reaches the preset thermal threshold, it can actively trigger the early warning linkage. Users can take the initiative to view live video or live examples of the behavior category, and respond in time according to the situation.

In actual situations, the alarm method in the embodiments of the present application can be widely used in various fields. Examples are as follows:

You can preset behavior types such as people running, people gathering and parade, people fighting, people falling to the ground, etc. When the heat of running behavior of people in a designated monitoring area suddenly rises and reaches the upper limit of the preset threshold of the behavior heat map system, an alert is triggered and the live video is pushed Or give the manager a short video of the on-site behavior. If you check that it is a fire in the shopping mall, you can quickly send a fire alarm to support; when the heat of the behavior of people falling to the ground or fighting with people in a certain area suddenly rises and reaches the upper threshold, the manager actually checks and finds In the event of a violent terrorist incident at the railway station, staff can be quickly sent to support.

The behavior types of people queuing, people staying, people dragging suitcases can be preset. When the heat of the people staying in a designated monitoring area suddenly rises and reaches the upper limit of the preset threshold, the manager checks the pushed live video or short video and finds the train If a large number of passengers are found in a square at a station, transportation resources or evacuation personnel can be quickly dispatched to evacuate the crowd.

You can preset behavior types such as students bowing their heads, students sleeping at the table, and students standing up and speaking. When the heat of the student sleeping at the table suddenly rises in the campus monitoring area (belonging to the normal teaching time period), the linkage strategy is triggered to alert the teaching administrator. The teaching manager checked the pushed video or short behavioral video and found that some classrooms had a low teaching atmosphere, and they could understand the teaching work in time and improve the quality of teaching.

You can preset behavior types such as cattle feeding, cattle drinking, cattle lying, and cattle violent exercise. When the heat of the cattle lying behavior in the pasture monitoring area suddenly rises (belonging to the normal eating period), the linkage strategy will be triggered to alert the pasture manager , After checking the pushed live video or short video by the ranch manager, it is found that the herd is suspected of being poisoned or epidemic, and the disease control and hygiene work can be carried out quickly.

In the embodiments of this application, it is possible to easily perform multi-functional combined use based on the statistical results of the behavior list of the designated monitoring area, including viewing the distribution characteristics of a single/any multiple behaviors, and viewing the behavior distribution characteristics of a single area/any multiple areas. The interactive operation is simple. The behavioral heat map is used for behavior preview and scheduling, which is convenient for users to quickly pay attention to the on-site situation, facilitate evidence collection, and quickly make system scheduling, which improves the level of intelligence.

In order to be able to detect multiple designated behaviors at the same time, in a possible implementation manner, the sub-regions of the above-mentioned behavior heat map include thermal colors, and the above-mentioned designated behaviors have multiple designated behaviors, and different designated behaviors correspond to different thermal colors. The intensity of the aforementioned thermal color is positively correlated with the frequency of occurrence of the specified behavior corresponding to the aforementioned thermal color, and each of the aforementioned thermal colors corresponds to a corresponding alarm linkage;

When the above-mentioned sub-area of the behavior heat map meets the preset alarm condition, triggering an alarm for the sub-area meeting the preset alarm condition includes:

Step 1: For each thermal color in each of the above-mentioned sub-regions, compare the depth of the thermal color with the size of the preset degree preset value corresponding to the thermal color.

Step 2: For the target thermal color whose depth is greater than the preset degree, trigger an alarm linkage for the sub-region where the target thermal color is located and corresponding to the target thermal color.

A preset degree threshold is set for each thermal color in advance, and the preset degree threshold for different thermal colors may be the same or different, and set according to actual needs. Different alarm linkages can be set for the thermal colors of different sub-regions, or the same alarm linkage can be set, which can be set according to actual needs. The back-end device analyzes the thermal colors of each sub-region respectively, and compares the intensity of the thermal color with the preset degree corresponding to the thermal color for any thermal color. When the depth of the thermal color is greater than the preset degree threshold corresponding to the thermal color, an alarm linkage for the sub-region where the thermal color is located and corresponding to the thermal color is executed.

In the embodiments of the present application, the detection and alarm of multiple specified behaviors can be simultaneously realized based on the behavior heat map, which can meet various needs of users.

Step 1: Obtain the user's display instruction for the sub-region to be displayed.

Step 2: According to the display instruction, display the image data in the sub-region to be displayed, wherein the image data in the sub-region to be displayed is the video stream of the monitoring area in the sub-region to be displayed.

The image data is the video stream of each monitoring area collected by the monitoring equipment, and the user can display the image data in the sub-area to be displayed through display instructions. In some cases, the sub-region to be displayed includes multiple image data, and a preview window of each image data may be generated first for the user to choose to display.

In the embodiments of the present application, the display of the image data of the actual monitoring scene is realized, which can help the user to fully understand the actual situation and meet the various needs of the user.

The embodiment of the present application also provides a method for sending a behavior list. See FIG. 6, which is applied to a front-end smart device. The method includes:

S601: Acquire image data of a preset monitoring area.

The behavior list sending method of the embodiment of the present application is applied to a front-end smart device, and therefore can be implemented by a front-end smart device. Specifically, the front-end smart device may be a smart camera or a hard disk video recorder. The smart camera can directly collect the image data of the preset monitoring area to obtain the image data of the preset monitoring area. The hard disk video recorder can obtain the image data of the preset monitoring area through the connected camera.

S602: Analyze the above-mentioned image data through computer vision technology to obtain a behavior analysis result of each designated target in the above-mentioned image data.

The front-end intelligent equipment recognizes the designated target in the image data through target detection and calculation, and tracks each designated target through the target tracking algorithm, so as to obtain the position of each designated target in the image data, according to the designated target in the image data Perform behavior recognition on each designated target and obtain the behavior analysis result of each designated target.

S603: According to the behavior analysis result of each of the specified targets, classify each of the specified targets to obtain multiple behavior lists, wherein the behavior types of the specified targets in the same behavior list are the same.

According to the behavior analysis results of the specified target, each specified target of the same behavior type is divided into a behavior list. In addition to recording the corresponding behavior type and the identification of the specified target, the behavior list can also record the location of the specified target, and specify the target. The location of can be the image data/preset monitoring area to which the specified target belongs, or the location of the specified target is the actual coordinates of the specified target, etc.

S604: Send each of the foregoing behavior lists.

The smart camera or hard disk video recorder sends each behavior list to the server so that the server generates a behavior heat map according to the behavior list. The generation process of the behavior heat map is the same as the above-mentioned behavior heat map generation method, and will not be repeated here.

Optionally, the behavior analysis result of each designated target is the behavior analysis result of each sampled designated target. The foregoing image data is analyzed through computer vision technology to obtain the behavior analysis result of each designated target in the image data, including:

Step 1: Determine each designated target in the image data and the position of each designated target through a preset target detection algorithm and a preset target tracking algorithm.

Through the target detection calculation, the designated target in the image data is recognized, and the designated target is tracked through the target tracking algorithm, so as to obtain the position of each designated target in the image data. In a possible implementation, in order to be effective To distinguish each designated target, you can set a unique ID for each designated target. The target detection algorithm may include pedestrian target detection, for example, HOG, DPM, FRCNN, YOLO, SSD, and the target tracking algorithm may be a multi-target tracking algorithm method.

The second step is to sample each designated target in the above-mentioned image data by using a preset target sampling algorithm to obtain each sampled designated target.

Target sampling for each specified target, for example, target sparse sampling of the specified target in each image data. Target sparse sampling methods include but are not limited to target uniform sampling, point uniform sampling, point weighted sampling, and area-based target quantity Sampling, etc., through sampling, an appropriate amount of designated targets of target scale can be obtained, that is, sampling designated targets.

Step 3: Perform target behavior sequence extraction on the image data according to the position of the designated target of each sample to obtain the pixel region sequence of the designated target of each sample.

In step one, the position of each designated target is determined, and each sampling designated target is a target in each designated target, so the position of each sampling designated target is known. The front-end smart device extracts the target behavior sequence of each image data according to the location of the designated target for each sample, and performs image interception from the image data according to a certain structure, such as Tubelet, etc., to obtain the pixel area sequence of the designated target for each sample.

Step 4: Analyze the pixel area sequence of each of the above-mentioned sampled designated targets to obtain the behavior analysis result of each of the above-mentioned sampled designated targets.

Analyze the pixel area sequence of the designated target for each sample, use the image sequence behavior recognition framework, such as LSTM, dual-stream network, C3D, P3D, ArtNet, PointNet, PointSIFT, etc., and combine the classification neural network to extract the sequence behavior feature to obtain each designated The behavior analysis result of the target pixel area sequence. Among them, the classification neural network includes but is not limited to Resnet18, Resnet50, Resnet101, Resnet152, Inception-v1, VGG, etc. In a possible implementation manner, the behavior analysis result includes behavior category and confidence.

The embodiment of the application provides an alarm device, which includes:

The heat map display module is used to display the behavior heat map of the area to be counted, wherein the behavior heat map represents the frequency of the specified behavior in each sub-region of the area to be counted;

The alarm trigger module is used to trigger an alarm for the sub-area meeting the preset alarm condition when the sub-area of the above-mentioned behavioral heat map meets the preset alarm condition.

Optionally, the above alarm trigger module includes:

The frequency comparison sub-module is used to compare the frequency of the specified behavior in each sub-region of the behavior heat map with the preset frequency threshold;

The sub-area alarm sub-module is used to trigger an alarm for the target sub-area for the target sub-area whose frequency of occurrence of the specified behavior is greater than the preset frequency threshold.

Optionally, the sub-regions of the behavior heat map include thermal colors, and the thermal colors represent the frequency of occurrence of the specified behaviors in the sub-regions, and the higher the frequency of the specified behaviors in the sub-regions, the lower the thermal color of the sub-regions. The higher the heating value;

The above alarm trigger module includes:

The thermal value comparison sub-module is used to compare the thermal value of the thermal color of each of the above sub-regions with the preset thermal pre-value;

The triggering alarm sub-module is used for triggering an alarm for the above-mentioned sub-area to be alarmed for the sub-area to be alarmed whose heating value is greater than the above-mentioned preset thermal threshold value.

Optionally, the sub-regions of the above-mentioned behavior heat map include thermal colors, the above-mentioned designated behaviors are multiple designated behaviors, and different designated behaviors correspond to different thermal colors. The depth of the above-mentioned thermal color corresponds to the designated behavior corresponding to the above-mentioned thermal color. The frequency is positively correlated, and each of the above thermal colors corresponds to the corresponding alarm linkage;

The above alarm trigger module is specifically used for:

For each thermal color in each of the aforementioned sub-regions, comparing the depth of the thermal color with the size of the preset degree preset value corresponding to the thermal color;

The image data display module is used to display the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is the video stream of the monitoring area in the sub-area to be displayed .

The embodiment of the present application also provides a device for generating a behavioral heat map. See FIG. 7, which is applied to a back-end device. The device includes:

The analysis result obtaining module 701 is configured to obtain the behavior analysis result of each designated target in the image data of each preset monitoring area, where the foregoing preset monitoring area is an area to be counted;

The sub-region frequency statistics module 702 is configured to determine the frequency of occurrence of the designated behavior in each sub-region of the area to be counted according to the behavior analysis result of each of the above-mentioned designated targets, wherein the above-mentioned sub-region and the above-mentioned preset monitoring area have an intersection;

The behavior heat map generating module 703 is configured to generate the behavior heat map of the designated behavior in the region to be counted according to the frequency of occurrence of the designated behavior in each of the above sub-regions.

Optionally, the aforementioned analysis result obtaining module 701 includes:

The behavior analysis sub-module is used to analyze each of the above-mentioned image data through computer vision technology to obtain the behavior analysis result of the designated target in each of the above-mentioned image data.

Optionally, the above behavior analysis sub-module includes:

The region sequence determination unit is used to track and detect the designated targets in each of the above-mentioned image data by computer vision technology, and extract the pixel region sequence of each of the above-mentioned designated targets;

The area sequence analysis unit is used to analyze the pixel area sequence of each of the designated targets to obtain the behavior analysis results of each of the designated targets.

Optionally, the pixel region sequence of each designated target is a pixel region sequence of each sample designated target, and the above-mentioned region sequence determining unit includes:

The position determination subunit is used to determine each designated target in each of the aforementioned image data and the position of each aforementioned designated target through a preset target detection algorithm and a preset target tracking algorithm;

The coefficient sampling subunit is used to sample each designated target in each of the above-mentioned image data by using a preset target sampling algorithm to obtain each sampled designated target;

The region intercepting and determining subunit is used to extract the target behavior sequence of each of the above-mentioned image data according to the position of each of the above-mentioned sample-designated targets to obtain the pixel region sequence of each of the above-mentioned sample-designated targets.

Optionally, the above-mentioned behavioral heat map generating device further includes:

The actual position acquisition module is used to acquire the actual position of each of the above-mentioned designated targets in the above-mentioned preset monitoring area;

The sub-region frequency statistics module is specifically configured to determine the frequency of occurrence of the designated behavior in each sub-region of the region to be counted based on the actual location of each designated target and the behavior analysis result of each designated target.

Optionally, the aforementioned actual location acquisition module includes:

The image position acquisition sub-module is used to determine the position of each designated target in the image data according to the pixel area sequence of each designated target;

The actual location mapping sub-module is used to determine the actual location of each specified target in the preset monitoring area according to the location of each specified target in the image data.

Optionally, the aforementioned sub-region frequency statistics module 702 includes:

The designated target classification sub-module is used to classify the designated targets according to the behavior analysis results of the designated targets to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The frequency determination sub-module is used to determine the frequency of occurrence of the above-mentioned designated behavior in each sub-region of the area to be counted according to the actual position of each designated target in the above-mentioned target behavior list.

The sub-region setting module is used to determine each sub-region in the area to be counted according to the above-mentioned granularity setting instruction.

Optionally, the aforementioned sub-area includes the aforementioned preset monitoring area, and the aforementioned sub-area frequency statistics module 702 includes:

An inclusion relationship determination sub-module for acquiring the inclusion relationship between each of the aforementioned sub-areas and each of the aforementioned preset monitoring areas;

The behavior frequency statistics sub-module is used to determine the frequency of occurrence of the specified behavior in each of the above sub-regions of the area to be counted according to the above inclusion relationship, the behavior analysis result of each of the above specified targets, and the preset monitoring area where each of the specified targets is located.

Optionally, the above-mentioned designated behaviors are multiple designated behaviors, and the above-mentioned behavior heat map generating module 703 includes:

The multi-frequency statistics sub-module is used to obtain the electronic map of the above-mentioned area to be counted, and obtain the frequency of occurrence of each designated behavior in each of the above-mentioned sub-regions;

The thermal color corresponding sub-module is used to determine the thermal color corresponding to each of the above specified behaviors;

The map coloring sub-module is used for any sub-area in the above electronic map, according to the frequency of occurrence of each specified behavior in the sub-area, and display the thermal color corresponding to each specified behavior in the sub-area on the map of the sub-area , Where the intensity of any thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.

The linkage strategy module is used to execute the linkage strategy of the specified behavior corresponding to the thermal color meeting the preset linkage rule when the thermal color of the specified monitoring area of the behavior heat map meets the preset linkage rule.

Optionally, the behavior analysis result of each designated target mentioned above is a list of designated targets that trigger each designated behavior; the above analysis result obtaining module includes:

The behavior list receiving sub-module is used to receive the behavior list sent by each smart device, wherein the above behavior list includes the identification of the designated target, and the behavior types of the designated targets in the same behavior list are the same;

The behavior list assembly sub-module is used to assemble each of the above-mentioned behavior lists to obtain the designated target lists that trigger each designated behavior.

An embodiment of the present application also provides an apparatus for sending a behavior list. See FIG. 8, which is applied to a front-end smart device. The apparatus includes:

The image data acquisition module 801 is used to acquire image data of a preset monitoring area;

The target behavior analysis module 802 is used to analyze the above-mentioned image data through computer vision technology to obtain the behavior analysis result of each specified target in the above-mentioned image data;

The designated target classification module 803 is configured to classify the designated targets according to the behavior analysis results of the designated targets to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The behavior list sending module 804 is configured to send each of the above-mentioned behavior lists.

Optionally, the behavior analysis result of each designated target is the behavior analysis result of each sampled designated target. The aforementioned target behavior analysis module 802 includes:

The designated target sampling sub-module is used to sample each designated target in the above-mentioned image data by using a preset target sampling algorithm to obtain each sampling designated target;

The pixel region interception sub-module is used to extract the target behavior sequence of the image data according to the position of the designated target of each of the above samples to obtain the pixel region sequence of each designated target of the sample;

The target behavior analysis sub-module is used to analyze the pixel area sequence of each of the above-mentioned sampling designated targets to obtain the behavior analysis results of each of the above-mentioned sampling designated targets.

The embodiment of the present application also provides an electronic device, including: a processor and a memory;

The aforementioned memory is used to store computer programs;

When the above-mentioned processor is used to execute the computer program stored in the above-mentioned memory, it realizes any one of the above-mentioned behavioral heat map generating methods.

Optionally, referring to FIG. 9, the electronic device of the embodiment of the present application further includes a communication interface 902 and a communication bus 904. The processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904. Specifically, the electronic device may be a server or a hard disk video recorder.

The aforementioned memory is used to store computer programs;

When the above-mentioned processor is used to execute the computer program stored in the above-mentioned memory, it realizes any one of the above-mentioned behavior list sending methods. Specifically, the electronic device can be a smart camera or a hard disk video recorder.

The aforementioned memory is used to store computer programs;

The processor is used to execute any of the above alarm methods when executing the computer program stored in the memory.

The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the aforementioned electronic device and other devices.

The memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located far away from the foregoing processor.

The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

An embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the above-mentioned behavioral heat map generation methods is implemented.

An embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the foregoing behavior list sending methods is implemented.

An embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the foregoing alarm methods is implemented.

It should be noted that, in this article, the technical features in each optional solution can be combined to form a solution as long as there is no contradiction, and these solutions are all within the scope of the disclosure of this application. Relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship between these entities or operations or order. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also includes Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, article, or equipment including the element.

The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the apparatus, electronic equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiments.

The above are only the preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in this application Within the scope of protection.

Claims

An alarm method, characterized in that the method includes:

Displaying the behavior heat map of the area to be counted, wherein the behavior heat map represents the frequency of occurrence of a specified behavior in each sub-region of the area to be counted;

When the sub-area of the behavior heat map meets the preset alarm condition, an alarm for the sub-area meeting the preset alarm condition is triggered.
The method according to claim 1, wherein the triggering an alarm for the sub-area meeting the preset alarm condition when the sub-area in the behavioral heat map meets the preset alarm condition comprises:

Respectively comparing the frequency of occurrence of the specified behavior in each sub-region of the behavior heat map with the magnitude of the preset frequency threshold;

For the target sub-region where the frequency of occurrence of the specified behavior is greater than the preset frequency threshold, trigger an alarm for the target sub-region.
The method according to claim 1, wherein the sub-regions of the behavior heat map include thermal colors, the thermal colors representing the frequency of occurrence of a specified behavior in the sub-region, and the specified behavior in the sub-region The higher the frequency of occurrence, the higher the thermal value of the thermal color of the sub-region;

The triggering an alarm for the sub-area meeting the preset alarm condition when the sub-area of the behavior heat map meets the preset alarm condition includes:

Respectively comparing the thermal value of the thermal color of each of the sub-regions with the preset thermal pre-value;

For the sub-area to be alarmed with a heating value greater than the preset heating threshold, an alarm for the sub-area to be alarmed is triggered.
The method according to claim 1, wherein the sub-regions of the behavior heat map include thermal colors, the designated behaviors are multiple designated behaviors, and different designated behaviors correspond to different thermal colors. The depth is positively correlated with the frequency of occurrence of the specified behavior corresponding to the thermal color, and each thermal color corresponds to a corresponding alarm linkage;

The triggering an alarm for the sub-area meeting the preset alarm condition when the sub-area of the behavior heat map meets the preset alarm condition includes:

For each thermal color in each of the sub-regions, comparing the depth of the thermal color with the size of the preset degree preset value corresponding to the thermal color;

For the target thermal color whose depth is greater than the preset degree preset value, trigger an alarm linkage for the sub-region where the target thermal color is located and corresponding to the target thermal color.
The method of claim 1, wherein the method further comprises:

Obtain the user's display instructions for the sub-areas to be displayed;

According to the display instruction, display the image data in the sub-region to be displayed, wherein the image data in the sub-region to be displayed is a video stream of the monitoring area in the sub-region to be displayed.
A method for generating a behavioral heat map, characterized in that it is applied to a back-end device, and the method includes:

Acquiring the behavior analysis result of each designated target in the image data of each preset monitoring area, where the preset monitoring area is an area in the area to be counted;

Determine the frequency of occurrence of the specified behavior in each sub-region of the area to be counted according to the behavior analysis result of each designated target, wherein the sub-region and the preset monitoring area have an intersection;

According to the frequency of occurrence of the designated behavior in each of the sub-regions, a behavior heat map of the designated behavior in the region to be counted is generated.
The method according to claim 6, wherein the obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area comprises:

Obtain image data of each preset monitoring area;

Determine each designated target in each of the image data and the position of each designated target through a preset target detection algorithm and a preset target tracking algorithm;

Sampling each designated target in each of the image data by a preset target sampling algorithm to obtain each sampling designated target;

Performing target behavior sequence extraction on each of the image data according to the position of each of the sampling designated targets to obtain the pixel area sequence of each of the sampling designated targets;

The pixel area sequence of each designated target is analyzed, and the behavior analysis result of each designated target is obtained.
The method according to claim 6, wherein one of the sub-regions includes at least one of the preset monitoring regions, and the determination is made in each sub-region of the region to be counted according to the behavior analysis result of each of the designated targets The frequency of the specified behavior, including:

Acquiring the inclusion relationship between each of the sub-regions and each of the preset monitoring regions;

According to the inclusion relationship, the behavior analysis result of each designated target, and the preset monitoring area where each designated target is located, the frequency of occurrence of the designated behavior in each of the sub-regions of the area to be counted is determined.
The method according to claim 6, characterized in that, after the obtaining the behavior analysis result of each designated target in the image data of each preset monitoring area, the method further comprises:

Acquiring the actual position of each designated target in the preset monitoring area;

The determining the frequency of occurrence of the designated behavior in each sub-region of the area to be counted according to the behavior analysis result of each designated target includes:

According to the actual location of each designated target and the behavior analysis result of each designated target, the frequency of occurrence of the designated behavior in each sub-region of the area to be counted is determined.
The method according to claim 9, wherein the determining the frequency of occurrence of the specified behavior in each sub-region of the area to be counted is based on the actual location of each of the specified targets and the behavior analysis result of each of the specified targets, include:

According to the behavior analysis result of each designated target, classify each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

Determine the target behavior list corresponding to the specified behavior;

According to the actual position of each designated target in the target behavior list, the frequency of occurrence of the designated behavior in each sub-region of the area to be counted is determined.
The method according to claim 6, characterized in that, before determining the frequency of occurrence of the designated behavior in each sub-region of the area to be counted according to the behavior analysis result of each designated target, the method further comprises:

Acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of the sub-region;

According to the granularity setting instruction, each sub-area in the area to be counted is determined.
The method according to claim 6, wherein the designated behavior is a plurality of designated behaviors, and the behavior heat map of the designated behavior in the region to be counted is generated according to the frequency of occurrence of the designated behavior in each of the sub-regions ,include:

Acquiring an electronic map of the area to be counted, and acquiring the frequency of occurrence of each designated behavior in each of the sub-regions;

Determine the thermal color corresponding to each specified behavior;

For any subregion in the electronic map, according to the frequency of occurrence of each designated behavior in the subregion, the thermal color corresponding to each designated behavior in the subregion is displayed on the map of the subregion, where any The intensity of the thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.
The method according to claim 6, wherein the behavior analysis result of each designated target is a list of designated targets that trigger each designated behavior; and the behavior analysis of each designated target in the image data of each preset monitoring area is obtained The results include:

Receiving each behavior list sent by each front-end smart device, where the behavior list includes the identification of the designated target, and the behavior types of the designated targets in the same behavior list are the same;

Assemble each of the behavior lists, and obtain the designated target lists that trigger each designated behavior.
A method for sending a behavior list, characterized in that it is applied to a front-end smart device, and the method includes:

Obtain image data of the preset monitoring area;

Analyze the image data by computer vision technology to obtain the behavior analysis result of each designated target in the image data;

According to the behavior analysis result of each designated target, each designated target is classified to obtain multiple behavior lists, wherein the behavior type of each designated target in the same behavior list is the same;

Send a list of each of the described actions.
The method according to claim 14, wherein the behavior analysis result of each designated target is the behavior analysis result of each sampled designated target, and the image data is analyzed by computer vision technology to obtain the image data The behavioral analysis results of each designated target in the following:

Determine each designated target and the position of each designated target in the image data by using a preset target detection algorithm and a preset target tracking algorithm;

Sampling each designated target in the image data by using a preset target sampling algorithm to obtain each sampling designated target;

Performing target behavior sequence extraction on the image data according to the positions of the designated targets of each sample to obtain a sequence of pixel regions of the designated targets of each sample;

Analyze the pixel area sequence of each of the sampled designated targets to obtain the behavior analysis result of each of the sampled designated targets.
An alarm device, characterized in that the device comprises:

The heat map display module is used to display the behavior heat map of the area to be counted, wherein the behavior heat map represents the frequency of occurrence of a specified behavior in each sub-region of the area to be counted;

The alarm triggering module is used to trigger an alarm for the sub-area meeting the preset alarm condition when the sub-area of the behavior heat map meets the preset alarm condition.
The device according to claim 16, wherein the alarm triggering module comprises:

A frequency comparison sub-module for comparing the frequency of occurrence of a specified behavior in each sub-region of the behavior heat map with a preset frequency threshold;

The sub-area alarm sub-module is used for triggering an alarm for the target sub-area for the target sub-area whose frequency of occurrence of the specified behavior is greater than the preset frequency threshold.
The device according to claim 16, wherein the sub-regions of the behavior heat map include thermal colors, and the thermal colors represent the frequency of occurrence of a specified behavior in the sub-region, and the specified behavior in the sub-region The higher the frequency of occurrence, the higher the thermal value of the thermal color of the sub-region;

The alarm trigger module includes:

The thermal value comparison sub-module is used to compare the thermal value of the thermal color of each of the sub-regions with the preset thermal value;

The triggering alarm sub-module is used for triggering an alarm for the sub-area to be alarmed whose heating value is greater than the preset heating threshold.
A behavioral heat map generating device, characterized in that it is applied to a back-end device, and the device includes:

The analysis result obtaining module is used to obtain the behavior analysis result of each designated target in the image data of each preset monitoring area, wherein the preset monitoring area is the area in the area to be counted;

The sub-region frequency statistics module is used to determine the frequency of occurrence of the designated behavior in each sub-region of the region to be counted according to the behavior analysis result of each designated target, wherein the sub-region and the preset monitoring region exist Intersection

The behavior heat map generating module is configured to generate a behavior heat map of the designated behavior of the region to be counted according to the frequency of occurrence of the designated behavior in each of the sub-regions.
The apparatus according to claim 19, wherein one of the sub-regions includes at least one of the preset monitoring regions, and the sub-region frequency statistics module includes:

An inclusion relationship determination sub-module for obtaining the inclusion relationship between each of the sub-regions and each of the preset monitoring regions;

The behavior frequency statistics sub-module is used to determine the occurrence of a designated behavior in each of the sub-regions of the area to be counted according to the inclusion relationship, the behavior analysis result of each of the designated targets, and the preset monitoring area where each designated target is The frequency.
The apparatus according to claim 19, wherein the sub-region frequency statistics module comprises:

The designated target classification sub-module is used to classify each designated target according to the behavior analysis result of each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The target list determination sub-module is used to determine the target behavior list corresponding to the specified behavior;

The frequency determination sub-module is used to determine the frequency of occurrence of the specified behavior in each sub-region of the area to be counted according to the actual position of each designated target in the target behavior list.
The device according to claim 19, wherein the device further comprises:

A setting instruction acquisition module, configured to acquire a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of the sub-region;

The sub-region setting module is used to determine each sub-region in the area to be counted according to the granularity setting instruction.
The device according to claim 19, wherein the designated behavior is a plurality of designated behaviors, and the behavior heat map generating module comprises:

The multi-frequency statistics sub-module is used to obtain an electronic map of the area to be counted, and obtain the frequency of occurrence of each designated behavior in each sub-area;

The thermal color corresponding sub-module is used to determine the thermal color corresponding to each specified behavior;

The map coloring sub-module is used for any sub-area in the electronic map, according to the frequency of occurrence of each specified behavior in the sub-area, display the corresponding to each specified behavior in the sub-area on the map of the sub-area Thermal color, where the intensity of any thermal color is positively related to the frequency of occurrence of the specified behavior corresponding to the thermal color.
The device according to claim 19, wherein the behavior analysis result of each designated target is a list of designated targets that trigger each designated behavior; and the analysis result obtaining module comprises:

The behavior list receiving sub-module is configured to receive the behavior list sent by each smart device, wherein the behavior list includes the identification of the designated target, and the behavior types of the designated targets in the same behavior list are the same;

The behavior list assembling sub-module is used to assemble each of the behavior lists to obtain the designated target lists that trigger each designated behavior.
A behavior list sending device, which is characterized in that it is applied to a front-end smart device, and the device includes:

The image data acquisition module is used to acquire the image data of the preset monitoring area;

The target behavior analysis module is used to analyze the image data through computer vision technology to obtain the behavior analysis result of each designated target in the image data;

The designated target classification module is configured to classify each designated target according to the behavior analysis result of each designated target to obtain multiple behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;

The behavior list sending module is used to send each of the behavior lists.
An electronic device, characterized in that it includes a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the alarm method according to any one of claims 1-5 when executing the program stored in the memory.
An electronic device, characterized in that it includes a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the method for generating a behavioral heat map according to any one of claims 6-13 when executing the program stored in the memory.
An electronic device, characterized in that it includes a processor and a memory;

The memory is used to store computer programs;

The processor is configured to implement the method for sending a behavior list according to any one of claims 14-15 when executing the program stored in the memory.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for generating a behavioral heat map according to any one of claims 6-13 is realized .
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for sending a behavior list according to any one of claims 14-15 is realized.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the alarm method according to any one of claims 1-5 is realized.