CN111862521A - Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium - Google Patents

Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111862521A
CN111862521A CN201910351634.4A CN201910351634A CN111862521A CN 111862521 A CN111862521 A CN 111862521A CN 201910351634 A CN201910351634 A CN 201910351634A CN 111862521 A CN111862521 A CN 111862521A
Authority
CN
China
Prior art keywords
behavior
sub
target
region
specified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910351634.4A
Other languages
Chinese (zh)
Other versions
CN111862521B (en
Inventor
赵飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201910351634.4A priority Critical patent/CN111862521B/en
Priority to PCT/CN2020/085423 priority patent/WO2020221031A1/en
Publication of CN111862521A publication Critical patent/CN111862521A/en
Application granted granted Critical
Publication of CN111862521B publication Critical patent/CN111862521B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00—Fire alarms; Alarms responsive to explosion
    • G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • G08B17/125—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00—Fire alarms; Alarms responsive to explosion
    • G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons

Abstract

The embodiment of the application provides a behavior thermodynamic diagram generation and alarm method, a behavior thermodynamic diagram generation and alarm device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a behavior analysis result of each designated target in image data of each preset monitoring area, wherein the preset monitoring area is an area in an area to be counted; determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein intersection exists between the sub-region and the preset monitoring region; and generating a behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region. According to the behavior thermodynamic diagram generation method, the occurrence frequency of the appointed behaviors of the sub-regions in the region to be counted is counted through the image data, the behavior thermodynamic diagram of the region to be counted is further generated, and the large-area region can be visually monitored.

Description

Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a behavior thermodynamic diagram generation and alarm method, apparatus, electronic device, and storage medium.
Background
With the development of computer vision technology, it becomes possible to automatically recognize a specified target by image data. In the related image recognition technologies, image data is analyzed by a computer vision technology, such as a convolutional neural network, so as to determine whether a target in the image data has a specified behavior. However, the above method only identifies whether the specified behavior occurs in the single-channel image data, which is inconvenient for intuitively monitoring a large area.
Disclosure of Invention
An embodiment of the application aims to provide a behavior thermodynamic diagram generation and alarm method, a behavior thermodynamic diagram generation and alarm device, electronic equipment and a storage medium, so that a large-area can be visually monitored. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present application provides an alarm method, where the method includes:
displaying a behavior thermodynamic diagram of a region to be counted, wherein the behavior thermodynamic diagram represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted;
and when the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition.
Optionally, when the sub-region in the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm for the sub-region meeting the preset alarm condition, including:
respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value;
and triggering an alarm aiming at the target sub-region with the frequency of the occurrence of the specified behaviors larger than the preset frequency threshold.
Optionally, a sub-region of the behavioral thermodynamic diagram includes a thermodynamic color, the thermodynamic color characterizes the frequency of occurrence of a specific behavior in the sub-region, and the higher the frequency of occurrence of the specific behavior in the sub-region is, the higher the thermodynamic value of the thermodynamic color of the sub-region is;
when the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition, wherein the alarm comprises the following steps:
respectively comparing the thermal force value of the thermal force color of each subarea with a preset thermal force preset value;
and triggering the alarm aiming at the subarea to be alarmed, wherein the heat value of the subarea to be alarmed is larger than the preset heat threshold value.
Optionally, the sub-region of the behavior thermodynamic diagram includes a thermal color, the specified behaviors include multiple specified behaviors, different specified behaviors correspond to different thermal colors, the depth of the thermal color is positively correlated with the frequency of occurrence of the specified behavior corresponding to the thermal color, and each thermal color corresponds to a corresponding alarm linkage;
When the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition, wherein the alarm comprises the following steps:
aiming at each thermal color in each sub-area, comparing the depth degree of the thermal color with a preset degree preset value corresponding to the thermal color;
and triggering alarm linkage aiming at the subarea where the target heating power color is located and corresponding to the target heating power color aiming at the target heating power color with the depth degree larger than the preset degree preset value.
Optionally, the alarm method according to the embodiment of the present application further includes:
acquiring a display instruction of a user for a sub-region to be displayed;
and displaying the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is a video stream of a monitoring area in the sub-area to be displayed.
In a second aspect, an embodiment of the present application provides a behavior thermodynamic diagram generation method, which is applied to a backend device, and the method includes:
acquiring a behavior analysis result of each designated target in image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted;
Determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein an intersection exists between the sub-region and the preset monitoring region;
and generating a behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region.
Optionally, the obtaining of the behavior analysis result of each designated target in the image data of each preset monitoring area includes:
acquiring image data of each preset monitoring area;
respectively tracking and detecting the designated targets in the image data through a computer vision technology, and extracting pixel region sequences of the designated targets;
and analyzing the pixel region sequence of each specified target to obtain a behavior analysis result of each specified target.
Optionally, the step of extracting the pixel area sequence of each designated target by tracking and detecting the designated target in each image data through a computer vision technique, where the pixel area sequence of each designated target is the pixel area sequence of each sampling designated target, includes:
determining each designated target and the position of each designated target in each image data through a preset target detection algorithm and a preset target tracking algorithm;
Sampling each designated target in each image data through a preset target sampling algorithm to obtain each sampling designated target;
and performing target behavior sequence extraction on the image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target.
Optionally, one of the sub-regions at least includes one preset monitoring region, and the determining, according to the behavior analysis result of each of the designated targets, the frequency of occurrence of the designated behavior in each sub-region of the region to be counted includes:
acquiring the inclusion relation between each sub-area and each preset monitoring area;
and determining the frequency of the designated behaviors in each sub-area of the area to be counted according to the inclusion relation, the behavior analysis result of each designated target and the preset monitoring area where each designated target is located.
Optionally, after the obtaining of the behavior analysis result of each designated target in the image data of each preset monitoring area, the method further includes:
acquiring the actual position of each designated target in the preset monitoring area;
determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein the determining comprises the following steps:
And determining the occurrence frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target.
Optionally, the obtaining the actual position of each specified target in the preset monitoring area includes:
determining the position of each specified target in the image data according to the pixel region sequence of each specified target;
and determining the actual position of each specified target in the preset monitoring area according to the position of each specified target in the image data.
Optionally, the determining, according to the actual position of each of the designated targets and the behavior analysis result of each of the designated targets, the frequency of occurrence of the designated behavior in each sub-region of the region to be counted includes:
classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
determining a target behavior list corresponding to the specified behavior;
and determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target in the target behavior list.
Optionally, before determining the frequency of occurrence of the specified behavior in each sub-region of the region to be counted according to the behavior analysis result of each specified target, the method further includes:
acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of a sub-region;
and determining each sub-area in the area to be counted according to the granularity setting instruction.
Optionally, the generating a behavior thermodynamic diagram of the specified behavior of the region to be counted according to the occurrence frequency of the specified behavior in each sub-region includes:
acquiring an electronic map of the area to be counted, and acquiring the frequency of each appointed behavior in each sub-area;
determining the thermal color corresponding to each designated behavior;
and aiming at any sub-area in the electronic map, displaying the thermal color corresponding to each specified behavior in the sub-area in the map of the sub-area according to the occurrence frequency of each specified behavior in the sub-area, wherein the shade degree of any thermal color is positively correlated with the occurrence frequency of the specified behavior corresponding to the thermal color.
Optionally, the behavior analysis result of each specified target is a specified target list for triggering each specified behavior; the acquiring of the behavior analysis result of each designated target in the image data of each preset monitoring area includes:
Receiving each behavior list sent by each front-end intelligent device, wherein the behavior lists comprise identifiers of designated targets, and the behavior types of the designated targets in the same behavior list are the same;
and assembling each behavior list to respectively obtain an appointed target list for triggering each appointed behavior.
In a third aspect, an embodiment of the present application provides a method for sending a behavior list, where the method is applied to a front-end intelligent device, and the method includes:
acquiring image data of a preset monitoring area;
analyzing the image data through a computer vision technology to obtain a behavior analysis result of each designated target in the image data;
classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
and sending each behavior list.
Optionally, the analyzing the behavior of each designated target is a behavior analysis result of each sampled designated target, and analyzing the image data by using a computer vision technique to obtain a behavior analysis result of each designated target in the image data includes:
Determining each designated target and the position of each designated target in the image data through a preset target detection algorithm and a preset target tracking algorithm;
sampling each designated target in the image data through a preset target sampling algorithm to obtain each sampling designated target;
according to the position of each sampling specified target, performing target behavior sequence extraction on the image data to obtain a pixel region sequence of each sampling specified target;
and analyzing the pixel region sequence of each sampling specified target to obtain a behavior analysis result of each sampling specified target.
In a fourth aspect, an embodiment of the present application provides an alarm device, where the alarm device includes:
the thermodynamic diagram display module is used for displaying a behavior thermodynamic diagram of the region to be counted, wherein the behavior thermodynamic diagram represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted;
and the alarm triggering module is used for triggering the alarm aiming at the subarea meeting the preset alarm condition when the subarea of the behavior thermodynamic diagram meets the preset alarm condition.
Optionally, the alarm triggering module includes:
the frequency comparison submodule is used for respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value;
And the sub-region alarm sub-module is used for triggering an alarm aiming at the target sub-region with the frequency of the occurrence of the specified behavior larger than the preset frequency threshold.
Optionally, a sub-region of the behavioral thermodynamic diagram includes a thermodynamic color, the thermodynamic color characterizes the frequency of occurrence of a specific behavior in the sub-region, and the higher the frequency of occurrence of the specific behavior in the sub-region is, the higher the thermodynamic value of the thermodynamic color of the sub-region is;
the alarm triggering module comprises:
the thermal force value comparison submodule is used for respectively comparing the thermal force value of the thermal force color of each subarea with the preset thermal force preset value;
and the triggering alarm submodule is used for triggering the alarm aiming at the to-be-alarmed subarea with the heat value larger than the preset thermal threshold value.
Optionally, the sub-region of the behavior thermodynamic diagram includes a thermal color, the specified behaviors include multiple specified behaviors, different specified behaviors correspond to different thermal colors, the depth of the thermal color is positively correlated with the frequency of occurrence of the specified behavior corresponding to the thermal color, and each thermal color corresponds to a corresponding alarm linkage;
The alarm triggering module is specifically used for:
aiming at each thermal color in each sub-area, comparing the depth degree of the thermal color with a preset degree preset value corresponding to the thermal color;
and triggering alarm linkage aiming at the subarea where the target heating power color is located and corresponding to the target heating power color aiming at the target heating power color with the depth degree larger than the preset degree preset value.
Optionally, the alarm device of the embodiment of the present application further includes:
the display instruction receiving module is used for acquiring a display instruction of a user for a sub-region to be displayed;
and the image data display module is used for displaying the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is a video stream of a monitoring area in the sub-area to be displayed.
In a fifth aspect, an embodiment of the present application provides a behavior thermodynamic diagram generation apparatus, which is applied to a backend device, and the apparatus includes:
the analysis result acquisition module is used for acquiring behavior analysis results of each designated target in the image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted;
The sub-region frequency counting module is used for determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein the sub-region and the preset monitoring region have intersection;
and the behavior thermodynamic diagram generating module is used for generating the behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region.
Optionally, the analysis result obtaining module includes:
the image data acquisition submodule is used for acquiring the image data of each preset monitoring area;
a behavior analysis submodule, the behavior analysis submodule comprising:
the region sequence determining unit is used for respectively tracking and detecting the specified targets in the image data through a computer vision technology and extracting pixel region sequences of the specified targets;
and the area sequence analysis unit is used for analyzing the pixel area sequence of each specified target to obtain a behavior analysis result of each specified target.
Optionally, the pixel area sequence of each designated target is a pixel area sequence of each sampling designated target, and the area sequence determining unit includes:
A position determining subunit, configured to determine, through a preset target detection algorithm and a preset target tracking algorithm, each of the designated targets in the image data and a position of each of the designated targets;
the coefficient sampling subunit is used for sampling each specified target in each image data through a preset target sampling algorithm to obtain each sampling specified target;
and the region interception determining subunit is used for performing target behavior sequence extraction on each image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target.
Optionally, one of the sub-regions at least includes one of the preset monitoring regions, and the sub-region frequency statistics module includes:
the inclusion relation determining submodule is used for acquiring the inclusion relation between each sub-area and each preset monitoring area;
and the behavior frequency counting submodule is used for determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the inclusion relationship, the behavior analysis result of each specified target and the preset monitoring area where each specified target is located.
Optionally, the behavior thermodynamic diagram generation apparatus according to the embodiment of the present application further includes:
An actual position obtaining module, configured to obtain actual positions of the designated targets in the preset monitoring area;
the sub-region frequency statistics module is specifically configured to: and determining the occurrence frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target.
Optionally, the actual position obtaining module includes:
an image position obtaining submodule, configured to determine, according to the pixel region sequence of each of the designated objects, a position of each of the designated objects in the image data;
and the actual position mapping submodule is used for determining the actual position of each specified target in the preset monitoring area according to the position of each specified target in the image data.
Optionally, the sub-region frequency statistics module includes:
the designated target classification submodule is used for classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
the target list determining submodule is used for determining a target behavior list corresponding to the specified behavior;
And the frequency determining submodule is used for determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target in the target behavior list.
Optionally, the behavior thermodynamic diagram generation apparatus according to the embodiment of the present application further includes:
the setting instruction acquisition module is used for acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of a sub-region;
and the sub-region setting module is used for determining each sub-region in the region to be counted according to the granularity setting instruction.
Optionally, the specific behavior includes a plurality of specific behaviors, and the behavior thermodynamic diagram generation module includes:
the multi-frequency counting submodule is used for acquiring the electronic map of the area to be counted and acquiring the frequency of each appointed behavior in each sub-area;
the thermal color corresponding submodule is used for determining the thermal color corresponding to each specified behavior;
and the map coloring submodule is used for displaying the thermal color corresponding to each appointed behavior in the sub-area in the map of the sub-area according to the frequency of the appointed behavior in the sub-area aiming at any sub-area in the electronic map, wherein the depth of any thermal color is positively correlated with the frequency of the appointed behavior corresponding to the thermal color.
Optionally, the behavior analysis result of each specified target is a specified target list for triggering each specified behavior; the analysis result acquisition module comprises:
the behavior list receiving submodule is used for receiving a behavior list sent by each intelligent device, wherein the behavior list comprises an identifier of a specified target, and the behavior types of the specified targets in the same behavior list are the same;
and the behavior list assembling submodule is used for assembling each behavior list and respectively obtaining a specified target list for triggering each specified behavior.
In a sixth aspect, an embodiment of the present application provides an activity list sending apparatus, which is applied to a front-end smart device, and the apparatus includes:
the image data acquisition module is used for acquiring image data of a preset monitoring area;
the target behavior analysis module is used for analyzing the image data through a computer vision technology to obtain a behavior analysis result of each designated target in the image data;
the specified target classification module is used for classifying the specified targets according to the behavior analysis result of each specified target to obtain a plurality of behavior lists, wherein the behavior types of the specified targets in the same behavior list are the same;
And the behavior list sending module is used for sending each behavior list.
Optionally, the behavior analysis result of each specified target is a behavior analysis result of each sampling specified target, and the target behavior analysis module includes:
the target position determining submodule is used for determining each designated target and the position of each designated target in the image data through a preset target detection algorithm and a preset target tracking algorithm;
the specified target sampling submodule is used for sampling each specified target in the image data through a preset target sampling algorithm to obtain each sampling specified target;
the pixel region intercepting submodule is used for extracting a target behavior sequence of the image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target;
and the target behavior analysis submodule is used for analyzing the pixel region sequence of each sampling specified target to obtain a behavior analysis result of each sampling specified target.
In a seventh aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a computer program;
The processor is configured to implement the alarm method according to any one of the first aspect described above when executing the program stored in the memory.
In an eighth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to implement the behavior thermodynamic diagram generation method according to any one of the second aspects when executing the program stored in the memory.
In a ninth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to implement the behavior list transmission method according to any one of the third aspects when executing the program stored in the memory.
In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the alarm method according to any one of the first aspect.
In an eleventh aspect, the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the behavior thermodynamic diagram generating method according to any one of the second aspects.
In a twelfth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the behavior list transmission method according to any one of the third aspects.
The behavior thermodynamic diagram generation and alarm method, the behavior thermodynamic diagram generation and alarm device, the electronic equipment and the storage medium, which are provided by the embodiment of the application, are used for acquiring behavior analysis results of each designated target in image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted; determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein intersection exists between the sub-region and the preset monitoring region; and generating a behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region. The occurrence frequency of the appointed behaviors of the sub-regions in the region to be counted is counted through the image data, and then the behavior thermodynamic diagram of the region to be counted is generated, so that the large-area region can be visually monitored. Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1a is a first schematic diagram of a behavior thermodynamic diagram generation method according to an embodiment of the present application;
FIG. 1b is a second schematic diagram of a method for generating a behavioral thermodynamic diagram according to an embodiment of the present application;
FIG. 2 is a third schematic diagram of a behavior thermodynamic diagram generation method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an alarm method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a deep learning algorithm training process according to an embodiment of the present application;
FIG. 5 is a fourth schematic diagram of a method for generating a behavioral thermodynamic diagram according to an embodiment of the present application;
fig. 6 is a schematic diagram of a behavior list transmission method according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a behavioral thermodynamic diagram generation apparatus according to an embodiment of the present application;
fig. 8 is a schematic diagram of a behavior list transmission apparatus according to an embodiment of the present application;
Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to facilitate a user to visually monitor a large area, an embodiment of the present application provides a behavior thermodynamic diagram generation method, which is applied to a backend device and includes:
s101, acquiring behavior analysis results of each designated target in image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted.
The behavior thermodynamic diagram generation method is applied to the back-end device, so that the behavior thermodynamic diagram generation method can be executed by the back-end device, and specifically, the back-end device can be a server, a personal computer, a hard disk video recorder or the like. The image data in the embodiment of the present application may be a video stream, and in some application scenes only including object recognition, may also be a single frame video frame.
The preset monitoring area is a monitoring area designated by a user in the area to be counted. The back-end equipment can directly acquire the behavior analysis result of each designated target in each image data through the intelligent cameras arranged in each preset monitoring area. The intelligent camera analyzes the image data acquired by the intelligent camera through a computer vision technology to obtain behavior analysis results of each designated target in the image data acquired by the intelligent camera.
Optionally, the smart camera sends the behavior analysis result of each designated target in a behavior list manner, for any smart camera, the smart camera establishes a behavior list for each behavior type, and the smart camera adds the identifier triggering the designated target of the designated behavior type to the corresponding behavior list. Correspondingly, the obtaining of the behavior analysis result of each designated target in the image data of each preset monitoring area includes: receiving a behavior list sent by each intelligent device, wherein the behavior list comprises an identifier of a specified target, and the behavior types of the specified targets in the same behavior list are the same; and assembling each behavior list to obtain a specified target list triggering each specified behavior respectively, wherein the behavior analysis result of each specified target is represented in a mode of triggering the specified target list of each specified behavior. The back-end equipment can summarize the analysis results of the image data of a plurality of preset monitoring areas by assembling the behavior lists sent by the intelligent equipment, and distributes the computing resources to the front-end intelligent equipment, so that the processing pressure of the back-end equipment is reduced, and the flexibility is improved. Specifically, the intelligent device here may be an intelligent video camera or a hard disk video recorder, etc.
In a possible embodiment, the image data analysis process is executed by a back-end device, and referring to fig. 1b, the obtaining of the behavior analysis result of each specified target in the image data of each preset monitoring area includes:
and S1011, acquiring image data of each preset monitoring area.
And the back-end equipment receives the image data of each preset monitoring area sent by each camera.
And S1012, analyzing the image data through a computer vision technology to obtain a behavior analysis result of the specified target in the image data.
And the back-end equipment obtains the behavior analysis result of the specified target in each image data through a computer vision technology. The designated target is a target that the user wishes to pay attention to, and may be a person, a vehicle, an animal, or the like, and may be specifically set according to actual requirements. In a possible implementation manner, the computer vision technology is a pre-trained deep learning algorithm, and the back-end device analyzes the image data by using the deep learning algorithm to obtain a behavior analysis result of a specified target in the image data. The process of the pre-trained deep learning algorithm may include, as shown in fig. 4: determining the interested behavior type, calibrating the behavior type of the specified target in each image data containing the specified target to obtain sample image data, inputting the sample image data into a deep learning algorithm for training, and obtaining a pre-trained deep learning algorithm after convergence.
Optionally, the analyzing, by using a computer vision technology, each of the image data to obtain a behavior analysis result of a specified target in each of the image data includes:
and step one, respectively tracking and detecting the specified targets in the image data through a computer vision technology, and extracting pixel area sequences of the specified targets.
The computer vision technology can comprise a target detection algorithm and a target tracking algorithm, the rear-end equipment respectively identifies the designated targets in the image data through target detection and calculation, tracks the designated targets through the target tracking algorithm to obtain the positions of the designated targets in the image data, and extracts pixel region sequences of the designated targets according to the positions of the designated targets in the image data.
In one possible embodiment, in order to reduce the processing pressure of the backend device, the pixel region sequence of each designated object is replaced by a pixel region sequence of the designated object by sampling. The tracking detection of the designated target in each image data by the computer vision technology to extract the pixel region sequence of each designated target includes:
And step A, determining each appointed target in the image data and the position of each appointed target through a preset target detection algorithm and a preset target tracking algorithm.
The back-end device identifies the designated targets in the image data through target detection and calculation, and tracks the designated targets through a target tracking algorithm, so as to obtain the positions of the designated targets in the image data. The object detection algorithm may include pedestrian object detection, for example, HOG (Histogram of Oriented Gradient), DPM (Deformable Parts Models), FRCNN (fast Regions with conditional Neural Networks, Faster partition-based convolutional Neural Networks), YOLO (You see Once), SSD (Single-point multi-box Detector), and the object tracking algorithm may be a multi-target tracking algorithm method.
And B, sampling each specified target in the image data through a preset target sampling algorithm to obtain each sampling specified target.
The back-end equipment also performs target sampling on each specified target, so that the processing pressure of the back-end equipment is reduced. The back-end device may sample the designated target through any relevant sampling algorithm. For example, target sparse sampling is performed on a specified target in each image data, a target sparse sampling method includes, but is not limited to, target uniform sampling, point weighted sampling, sampling based on the number of regional targets, and the like, and a specified target with a proper target scale, that is, a sampled specified target, can be obtained through sampling.
And step C, performing target behavior sequence extraction on the image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target.
In step a, the positions of the designated objects are determined, and the positions of the designated objects are known because the designated objects are all the objects in the designated objects. The back-end equipment extracts the target behavior sequence of each image data according to the position of each sampling designated target, and cuts the image from the image data according to a certain structure, such as a tube let, and the like, so as to obtain the pixel area sequence of each sampling designated target. The pixel region sequence of each designated object is replaced by a pixel region sequence of the designated object by sampling.
In the embodiment of the application, the pixel region sequence of each designated target is sampled, sparse sampling is completed according to the target density, the distribution characteristics of the designated targets in different preset monitoring regions are kept, the data volume of behavior type identification processing is reduced, and the practicability of the whole scheme is improved.
And step two, analyzing the pixel region sequence of each specified target to obtain a behavior analysis result of each specified target.
Analyzing the pixel region sequences of each designated target, and performing sequence behavior feature extraction by using an image sequence behavior recognition framework, for example, LSTM (Long Short-Term Memory), a double-current network, C3D (3D ConvNets, deep 3-dimensional convolution Networks), P3D (Pseudo-three-dimensional Residual Networks), ArtNet, PointNet, PointSIFT, and the like, in combination with a classification neural network, to obtain behavior analysis results of the pixel region sequences of each designated target. The classified Neural Network includes, but is not limited to, Network18 (Residual Neural Network18 ), Network50 (Residual Neural Network50, Residual Neural Network 50), Network101 (Residual Neural Network101 ), Network152 (Residual Neural Network152, Residual Neural Network 152), inclusion-v 1, VGG (Visual Geometry group Network), and the like. In one possible implementation, the behavior analysis results include a behavior category and a confidence level.
S102, determining the frequency of the occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein the sub-region and the preset monitoring region have intersection;
the region to be counted may be a preset region or a region designated by a user. In a possible implementation manner, before S102, the method further includes: acquiring a region selection instruction to be counted input by a user; and determining the area to be counted according to the area to be counted selecting instruction. And the selection instruction of the area to be counted represents the range of the area to be counted. Each sub-region of the region to be counted may be predetermined, for example, a plurality of area intervals are divided in advance according to the area size, and the size and the dividing method of the sub-region are set for each area interval. Determining the area interval of the region to be counted according to the size of the region to be counted, and determining each sub-region in the region to be counted according to the size of the sub-region corresponding to the area interval of the region to be counted and the dividing method. Of course, a fixed sub-region size and a division method may also be set, and for the region to be counted, each sub-region of the region to be counted is determined according to the fixed sub-region size and the division method.
In one possible embodiment, the sub-area is divided in advance, and the sub-areas are divided in advance according to different granularities, such as roads, floors, cells, urban areas, cities or provinces. And determining the sub-regions included in the region to be counted according to the pre-divided sub-regions.
In one possible implementation, different granularities may also be selected according to user requirements. Optionally, before determining the frequency of occurrence of the specified behavior in each sub-region of the region to be counted according to the behavior analysis result of each specified target, the method further includes:
step one, a granularity setting instruction input by a user is obtained, wherein the granularity setting instruction represents the size attribute of a sub-area. The granularity setting instruction characterizes the size of the sub-region, for example, the granularity setting instruction characterizes the sub-region as a road, a floor, a cell, an urban area, a city or a province, and the like.
And step two, determining each sub-area in the area to be counted according to the granularity setting instruction. For example, when the granularity setting instruction represents that the sub-regions are cells, determining each sub-region as a cell; and when the granularity setting instruction represents that the sub-regions are streets, determining that each sub-region is a street.
Through granularity setting, behavior analysis results of different granularities can be summarized, visual display of regional behaviors can be carried out with different colors and color shades by visually combining the electronic map, and the method is visual and easy to use.
The designated behavior may be a preset behavior type or a behavior type selected by the user in real time. In a possible embodiment, the method further includes: acquiring a designated behavior selection instruction input by a user, wherein the designated behavior selection instruction represents a behavior type of a designated behavior; and selecting an instruction according to the specified behavior, and determining the specified behavior.
The back-end equipment determines sub-areas where the designated targets are located according to the preset monitoring areas where the designated targets belong; and respectively counting the occurrence frequency of the specified behaviors in each sub-region according to the behavior analysis result of each specified target. In a possible implementation manner, the determining, according to the behavior analysis result of each of the designated targets, the frequency of occurrence of the designated behavior in each of the sub-regions of the region to be counted includes:
Step one, acquiring the inclusion relation between each sub-area and each preset monitoring area.
And respectively determining the preset monitoring areas included by the sub-areas according to the positions of the sub-areas and the positions of the preset monitoring areas.
And step two, determining the frequency of the appointed behaviors in each sub-area of the area to be counted according to the inclusion relation, the behavior analysis result of each appointed target and the preset monitoring area where each appointed target is located.
The image data is a video image of a preset monitoring area, and the designated target in any image data is the designated target in the preset monitoring area corresponding to the image data. If the sub-area comprises a preset monitoring area, the designated target in the preset monitoring area is the designated target in the sub-area. And respectively counting the occurrence frequency of the specified behaviors in each sub-region according to the behavior analysis result of each specified target.
In order to facilitate statistics of the behavior analysis result of the specific target, in a possible implementation, the method further includes: and classifying the designated targets according to the behavior analysis results of the designated targets to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same. According to the behavior analysis result of the designated target, dividing each designated target of the same behavior type into a behavior list, wherein the behavior list can record the corresponding behavior type and the mark of the designated target, and also can record the position of the designated target, and the position of the designated target can be image data/a preset monitoring area to which the designated target belongs, or the position of the designated target is the actual coordinate of the designated target, and the like.
And S103, generating a behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region.
And the back-end equipment colors each sub-region of the region to be counted in the electronic map according to the occurrence frequency of the specified behaviors in each sub-region, so as to obtain the behavior thermodynamic diagram of the specified behaviors in the region to be counted. In one possible embodiment, the frequency of occurrence of a given behavior in a sub-region may be represented by a cool-warm color, for example, the color of the sub-region tends to be warmer the higher the frequency of occurrence of the given behavior in the sub-region; the lower the frequency with which a given behavior in a sub-region occurs, the closer the color of the sub-region approaches a cool color.
In a possible implementation manner, the generating a behavior thermodynamic diagram of the specified behavior of the region to be counted according to the occurrence frequency of the specified behavior in each sub-region includes:
step one, acquiring an electronic map of the area to be counted, and acquiring the frequency of occurrence of each appointed behavior in each sub-area.
And step two, determining the thermal color corresponding to each appointed behavior.
The designated behavior includes a plurality of designated behaviors, and different thermal colors can be set for different designated behaviors. The thermal color corresponding to each designated behavior may be determined randomly or may be designated by the user, and is not described herein again.
And step three, aiming at any sub-area in the electronic map, displaying the thermal color corresponding to each appointed behavior in the sub-area in the map of the sub-area according to the frequency of the appointed behaviors in the sub-area, wherein the depth of any thermal color is positively correlated with the frequency of the appointed behaviors corresponding to the thermal color.
And displaying the thermal color of the specified behavior contained in any sub-area at the position of the electronic map of the sub-area. And the higher the frequency of the specified behaviors in the sub-area is, the deeper the thermal color depth corresponding to the specified behaviors is.
The method comprises the steps of using an electronic map as a substrate, using the color depth to represent the height of the frequency of the specified behaviors, using different colors to represent different behavior types, and displaying corresponding thermal colors at the positions of all sub-areas in the electronic map to obtain a behavior thermodynamic diagram of the area to be counted. In one possible implementation, the behavioral thermodynamic diagram may be scaled up or down, and the frequency statistics updated according to an electronic map scale. The user can select the image data in the sub-area from the electronic map to perform video preview, and the actual situation can be observed more truly. Optionally, the method further includes acquiring an image display instruction for a specified preset monitoring area; and displaying the image data of the appointed preset monitoring area according to the image display instruction. For example, a user may click a designated preset monitoring area in the sub-areas through a mouse or a touch screen, and the back-end device displays image data of the designated preset monitoring area after detecting a click instruction for the designated preset monitoring area.
In the embodiment of the application, the occurrence frequency of the appointed behaviors of each sub-region in the region to be counted is counted through the image data, so that the behavior thermodynamic diagram of the region to be counted is generated, and the large-area region can be visually monitored.
When the granularity requirement of the behavior thermodynamic diagram is small, the position of the specified target needs to be further positioned. In a possible implementation manner, after obtaining the behavior analysis result of each specified target in the image data of each preset monitoring area, the method further includes:
and acquiring the actual position of each specified target in the preset monitoring area.
The actual position of the designated target can be reported by front-end intelligent equipment such as an intelligent camera and the like, and can also be determined by back-end equipment according to image data.
In a possible implementation manner, referring to fig. 2, the acquiring an actual position of each of the designated objects in the preset monitoring area includes:
s201, determining the position of each of the designated objects in the image data according to the pixel region sequence of each of the designated objects.
The pixel region of the designated object may be a pixel region selected in an object frame of the designated object, and the position sequence of the designated object in the image data may be determined according to the pixel region sequence of the designated object, and may be, for example, a position coordinate sequence (a plurality of coordinate regions which are continuous in time sequence).
And S202, determining the actual position of each designated object in the preset monitoring area according to the position of each designated object in the image data.
And converting the position of the specified target in the image data into the actual position of the specified target in the preset monitoring area by a related coordinate conversion method, wherein the actual position can be a global positioning system coordinate or a self-defined area coordinate.
In a possible implementation manner, the actual position of each designated target may be sent to a back-end device by a front-end device such as a smart camera, and the back-end device may directly acquire the actual position of each designated target.
The determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target includes:
and determining the occurrence frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target.
And respectively determining the actual position of each sub-area of the area to be counted. And respectively determining the occurrence frequency of the specified behaviors in each sub-area according to the actual position of each specified target, the actual position of each sub-area and the behavior analysis result of each specified target.
In the embodiment of the application, the actual positions of the designated targets are determined, so that the method can be applied to the condition that the sub-region does not contain a complete preset monitoring region, even the condition that the sub-region is smaller than the preset monitoring region, and can be applied to the scene with smaller granularity of the behavior thermodynamic diagram, namely the scene with the smaller sub-region, the theoretical sub-region can be a minimum coordinate point, and the monitoring precision of the behavior thermodynamic diagram can be greatly improved.
In order to facilitate statistics of the behavior analysis results of the designated targets, in a possible implementation manner, the determining the frequency of occurrence of the designated behavior in each sub-area of the area to be counted according to the actual position of each designated target and the behavior analysis result of each designated target includes:
and S1021, classifying the designated targets according to behavior analysis results of the designated targets to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same.
And dividing the designated targets of the same behavior type into a behavior list according to the behavior analysis result of the designated targets. For any behavior list, the behavior type of the behavior list, the identification of each specified target contained in the behavior list, and the actual position of each specified target contained in the behavior list are recorded in the behavior list.
S1022, determining a target behavior list corresponding to the specified behavior.
And determining a behavior list corresponding to the specified behavior, namely a target behavior list.
And S1023, determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual positions of the specified targets in the target behavior list.
And respectively determining the actual position of each sub-area of the area to be counted. And respectively determining the occurrence frequency of the specified targets in each sub-region, namely the occurrence frequency of the specified behaviors in each sub-region according to the actual positions of the specified targets and the actual positions of the sub-regions in the target behavior list.
In the embodiment of the application, the behavior list is set, statistics of the behavior analysis result of the specified target is facilitated, and the generation efficiency of the behavior thermodynamic diagram is high.
An embodiment of the present application further provides an alarm method, referring to fig. 3, where the method includes:
s301, displaying a behavior thermodynamic diagram of the region to be counted, wherein the behavior thermodynamic diagram represents the frequency of the specified behavior in each sub-region of the region to be counted.
The alarm method of the embodiment of the application can be executed through the back-end equipment, and specifically, the back-end equipment can be a server, a personal computer or a hard disk video recorder and the like. The behavior thermodynamic diagrams can be obtained by any one of the above behavior thermodynamic diagram generation methods, and are not described herein again.
And S302, when the sub-area of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-area meeting the preset alarm condition.
The preset alarm condition may be set according to actual conditions, for example, the frequency of the designated action is set to be greater than a preset frequency threshold, or the thermal value of the thermal color is greater than a preset thermal preset value. In a possible embodiment, the triggering an alarm for a sub-area satisfying a preset alarm condition when the sub-area in the behavior thermodynamic diagram satisfies the preset alarm condition includes: respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value; and triggering an alarm aiming at the target sub-region with the frequency of the occurrence of the specified behaviors larger than the preset frequency threshold.
In a possible embodiment, the sub-regions of the behavior thermodynamic diagram include thermal colors, the thermal colors represent the frequency of occurrence of the specified behaviors in the sub-regions, and the higher the frequency of occurrence of the specified behaviors in the sub-regions is, the higher the thermal value of the thermal colors of the sub-regions is; the above triggering an alarm for a sub-area satisfying a preset alarm condition when the sub-area of the behavior thermodynamic diagram satisfies the preset alarm condition includes: respectively comparing the thermal force value of the thermal force color of each subarea with the preset thermal force preset value; and triggering the alarm aiming at the subarea to be alarmed, wherein the heat value of the subarea to be alarmed is larger than the preset heat threshold value.
The behavior thermodynamic diagram generation method according to the embodiment of the present application may be specifically as shown in fig. 5. The user can set concerned behavior types, the back-end equipment monitors the frequency of each appointed behavior in the area to be counted in real time, and early warning linkage can be triggered actively once the heat of the appointed behavior types obviously rises to reach a preset heat threshold value. The user can actively check the live video or the live behavior category sample and timely respond according to the situation.
In practical situations, the alarm method of the embodiment of the application can be widely applied to various fields. Examples are as follows:
the method can preset behavior types of personnel running, personnel gathering and walking, personnel fighting and falling down and the like, when the heat of the personnel running behavior in a certain specified monitoring area suddenly rises and reaches the upper limit of a preset threshold value of a behavior thermodynamic diagram system, early warning is triggered, a field video or a field behavior short video is pushed to a manager, and if the fact that a fire disaster occurs in a market is checked and found, fire alarm support can be quickly sent out; when the heat of the behavior of falling over the ground of people and the behavior of putting up the frame of people in a certain area suddenly rises to reach the upper limit of the threshold value, the manager actually checks and finds that the sudden terrorist event occurs in the railway station, and then the manager can quickly dispatch the staff for support.
Can predetermine personnel and queue up, personnel are detained, personnel drag action types such as suitcase, detain action heating power and rise suddenly when certain appointed monitoring area personnel, reach and predetermine the threshold value upper limit, look over the on-the-spot video or the short video of propelling movement through the administrator, discover a large amount of passengers in certain square of railway station discovery, then can dispatch transportation resources fast or dredge personnel and go to sparse crowd.
The student can be preset and the student is low, the student lies prone the table and sleeps, the student stands up the action type such as speech, when student lies prone the table sleep action heating power in campus monitoring area and rises suddenly (belong to normal teaching time quantum), trigger the linkage strategy, report an emergency and ask for help or increased vigilance to the teaching supervisor, the teaching supervisor looks over the video of propelling movement or the short video discovery of action, the phenomenon that teaching atmosphere is low has appeared in the individual classroom, can in time know teaching work, promote teaching quality.
The method can preset behavior types of feeding of the cattle, drinking of the cattle, lying areas of the cattle, violent movement of the cattle and the like, when the behavior heat of the lying areas of the cattle in a pasture monitoring area suddenly rises (belongs to a normal feeding time period), a linkage strategy is triggered, an alarm is given to a pasture manager, pushed field videos or short videos are checked by the pasture manager, poisoning or epidemic situations of the cattle are found, and then disease control sanitary work can be rapidly carried out.
In the embodiment of the application, the multifunctional combination can be simply and conveniently used according to the statistical result of the behavior list of the specified monitoring area, the distribution characteristics of single/any multiple behaviors and the behavior distribution characteristics of the single/any multiple areas are checked, and the user interaction operation is simple. The behavior thermodynamic diagram is adopted to preview and schedule the behaviors, so that a user can conveniently and quickly pay attention to the field condition, evidence can be conveniently obtained, system scheduling can be quickly carried out, and the intelligent level is improved.
In order to detect multiple designated behaviors simultaneously, in one possible implementation manner, a sub-region of the behavior thermodynamic diagram includes a thermal color, the designated behaviors include multiple designated behaviors, different designated behaviors correspond to different thermal colors, the depth of the thermal color is positively correlated with the occurrence frequency of the designated behavior corresponding to the thermal color, and each thermal color corresponds to a corresponding alarm linkage;
the above triggering an alarm for a sub-area satisfying a preset alarm condition when the sub-area of the behavior thermodynamic diagram satisfies the preset alarm condition includes:
step one, aiming at each thermal color in each sub-area, comparing the depth degree of the thermal color with the preset degree preset value corresponding to the thermal color.
And step two, triggering alarm linkage aiming at the subarea where the target heating power color is located and corresponding to the target heating power color aiming at the target heating power color with the depth degree larger than the preset degree preset value.
The preset degree threshold values are preset for all the thermal colors in advance, and the preset degree threshold values of different thermal colors can be the same or different and are set according to actual requirements. Different alarm linkage can be set for the heating power colors of different subregions, the same alarm linkage can also be set, and the setting is specifically carried out according to actual requirements. And the rear-end equipment analyzes each thermal color of each subregion respectively, and compares the depth degree of the thermal color with the preset degree corresponding to the thermal color according to any thermal color. And when the depth degree of the thermal color is greater than the preset degree threshold value corresponding to the thermal color, executing alarm linkage aiming at the subarea where the thermal color is located and corresponding to the thermal color.
In the embodiment of the application, detection and alarm of various specified behaviors can be simultaneously realized based on the behavior thermodynamic diagram, and various requirements of users can be met.
Optionally, the alarm method according to the embodiment of the present application further includes:
Step one, obtaining a display instruction of a user aiming at a sub-region to be displayed.
And secondly, displaying the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is the video stream of the monitoring area in the sub-area to be displayed.
The image data is a video stream of each monitoring area acquired by the monitoring equipment, and the user can display the image data in the sub-area to be displayed through the display instruction. In some cases, the sub-region to be displayed includes a plurality of image data, and a preview window of each image data may be generated first for the user to select to display.
According to the embodiment of the application, the display of the image data of the actual monitoring scene is realized, the user can be helped to know the actual situation more sufficiently, and various requirements of the user are met.
An embodiment of the present application further provides a method for sending a behavior list, see fig. 6, which is applied to a front-end intelligent device, and the method includes:
s601, acquiring image data of a preset monitoring area.
The behavior list sending method is applied to the front-end intelligent device, so that the method can be realized through the front-end intelligent device, and specifically, the front-end intelligent device can be an intelligent video camera or a hard disk video recorder and the like. The intelligent camera can directly collect the image data of the preset monitoring area, so that the image data of the preset monitoring area is obtained. The hard disk video recorder can acquire image data of a preset monitoring area through the connected camera.
And S602, analyzing the image data through a computer vision technology to obtain behavior analysis results of each designated target in the image data.
The front-end intelligent equipment identifies the designated targets in the image data through target detection and calculation, tracks all the designated targets through a target tracking algorithm to obtain the positions of all the designated targets in the image data, and identifies the behaviors of all the designated targets according to the positions of all the designated targets in the image data to obtain the behavior analysis results of all the designated targets.
S603, classifying the designated targets according to the behavior analysis results of the designated targets to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same.
According to the behavior analysis result of the designated target, dividing each designated target of the same behavior type into a behavior list, wherein the behavior list can record the corresponding behavior type and the mark of the designated target, and also can record the position of the designated target, and the position of the designated target can be image data/a preset monitoring area to which the designated target belongs, or the position of the designated target is the actual coordinate of the designated target, and the like.
S604, sending each behavior list.
The intelligent camera or the hard disk video recorder sends each behavior list to the server so that the server generates the behavior thermodynamic diagram according to the behavior list, and the generation process of the behavior thermodynamic diagram is as described above in the behavior thermodynamic diagram generation method, and details are not repeated here.
Optionally, the analyzing the image data by the computer vision technology to obtain the behavior analysis result of each designated target in the image data includes:
determining each designated target and the position of each designated target in the image data through a preset target detection algorithm and a preset target tracking algorithm.
In a possible embodiment, in order to effectively distinguish the designated targets, a unique ID may be set for each designated target. The target detection algorithm may include pedestrian target detection, e.g., HOG, DPM, FRCNN, YOLO, SSD, and the target tracking algorithm may be a multi-target tracking algorithm method.
And step two, sampling each designated target in the image data through a preset target sampling algorithm to obtain each sampling designated target.
Target sampling is performed on each designated target, for example, target sparse sampling is performed on the designated target in each image data, the target sparse sampling method includes, but is not limited to, target uniform sampling, point weighted sampling, sampling based on the number of regional targets, and the like, and a proper amount of designated targets in a target scale, that is, the designated targets are sampled.
And step three, performing target behavior sequence extraction on the image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target.
In the first step, the position of each designated target is determined, and each sampling designated target is a target in each designated target, so that the position of each sampling designated target is known. The front-end intelligent device extracts the target behavior sequence of each image data according to the position of each sampling designated target, and performs image interception from the image data according to a certain structure, such as a tube let and the like, to obtain the pixel area sequence of each sampling designated target.
And step four, analyzing the pixel region sequence of each sampling specified target to obtain a behavior analysis result of each sampling specified target.
Analyzing the pixel region sequence of each sampling designated target, and extracting sequence behavior characteristics by using an image sequence behavior identification framework, such as LSTM, a double-current network, C3D, P3D, ArtNet, PointNet, PointSIFT and the like, in combination with a classification neural network to obtain a behavior analysis result of the pixel region sequence of each designated target. The classified neural network includes, but is not limited to, Resnet18, Resnet50, Resnet101, Resnet152, inclusion-v 1, VGG, etc. In one possible implementation, the behavior analysis results include a behavior category and a confidence level.
The embodiment of the application provides an alarm device, and the device includes:
the thermodynamic diagram display module is used for displaying a behavior thermodynamic diagram of a region to be counted, wherein the behavior thermodynamic diagram represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted;
and the alarm triggering module is used for triggering the alarm aiming at the subarea meeting the preset alarm condition when the subarea of the behavior thermodynamic diagram meets the preset alarm condition.
Optionally, the alarm triggering module includes:
the frequency comparison submodule is used for respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value;
and the sub-region alarm sub-module is used for triggering an alarm aiming at the target sub-region of which the frequency of the occurrence of the specified behavior is greater than the preset frequency threshold.
Optionally, the sub-region of the behavior thermodynamic diagram includes a thermal color, the thermal color represents the frequency of occurrence of the specified behavior in the sub-region, and the higher the frequency of occurrence of the specified behavior in the sub-region is, the higher the thermal value of the thermal color of the sub-region is;
the alarm triggering module comprises:
the thermal power value comparison submodule is used for respectively comparing the thermal power value of the thermal color of each subarea with the preset thermal power preset value;
and the triggering alarm submodule is used for triggering the alarm aiming at the to-be-alarmed subarea with the heat value larger than the preset heat threshold value.
Optionally, the sub-region of the behavior thermodynamic diagram includes a thermal color, the specified behavior includes a plurality of specified behaviors, different specified behaviors correspond to different thermal colors, the depth of the thermal color is positively correlated with the frequency of occurrence of the specified behavior corresponding to the thermal color, and each thermal color corresponds to a corresponding alarm linkage;
The alarm triggering module is specifically configured to:
aiming at each thermal color in each sub-area, comparing the depth degree of the thermal color with the preset degree preset value corresponding to the thermal color;
and triggering alarm linkage aiming at the subarea where the target heating power color is located and corresponding to the target heating power color aiming at the target heating power color with the depth degree larger than the preset degree preset value.
Optionally, the alarm device of the embodiment of the present application further includes:
the display instruction receiving module is used for acquiring a display instruction of a user for a sub-region to be displayed;
and the image data display module is used for displaying the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is a video stream of a monitoring area in the sub-area to be displayed.
An embodiment of the present application further provides a behavior thermodynamic diagram generation apparatus, see fig. 7, applied to a backend device, where the apparatus includes:
an analysis result obtaining module 701, configured to obtain a behavior analysis result of each designated target in image data of each preset monitoring area, where the preset monitoring area is an area to be counted;
A sub-region frequency statistics module 702, configured to determine, according to a behavior analysis result of each of the designated targets, a frequency of occurrence of a designated behavior in each sub-region of a region to be counted, where an intersection exists between the sub-region and the preset monitoring region;
a behavior thermodynamic diagram generating module 703, configured to generate a behavior thermodynamic diagram of the specified behavior of the region to be counted according to the frequency of occurrence of the specified behavior in each of the sub-regions.
Optionally, the analysis result obtaining module 701 includes:
the image data acquisition submodule is used for acquiring the image data of each preset monitoring area;
and the behavior analysis submodule is used for analyzing the image data through a computer vision technology to obtain a behavior analysis result of the specified target in the image data.
Optionally, the behavior analysis sub-module includes:
a region sequence determining unit, configured to perform tracking detection on the designated targets in the image data respectively through a computer vision technique, and extract pixel region sequences of the designated targets;
and the area sequence analysis unit is used for analyzing the pixel area sequence of each specified target to obtain a behavior analysis result of each specified target.
Optionally, the pixel area sequence of each designated target is a pixel area sequence of each sampling designated target, and the area sequence determining unit includes:
a position determining subunit, configured to determine, through a preset target detection algorithm and a preset target tracking algorithm, each designated target in each piece of image data and a position of each designated target;
the coefficient sampling subunit is used for sampling each specified target in each image data through a preset target sampling algorithm to obtain each sampling specified target;
and the region interception determining subunit is used for performing target behavior sequence extraction on each image data according to the position of each sampling specified target to obtain a pixel region sequence of each sampling specified target.
Optionally, the behavior thermodynamic diagram generating device further includes:
an actual position obtaining module, configured to obtain an actual position of each specified target in the preset monitoring area;
the sub-region frequency statistics module is specifically configured to: and determining the occurrence frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target.
Optionally, the actual position obtaining module includes:
an image position obtaining sub-module, configured to determine, according to a pixel region sequence of each of the designated objects, a position of each of the designated objects in the image data;
and the actual position mapping submodule is used for determining the actual position of each specified target in the preset monitoring area according to the position of each specified target in the image data.
Optionally, the sub-region frequency statistics module 702 includes:
the designated target classification submodule is used for classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
the target list determining submodule is used for determining a target behavior list corresponding to the specified behavior;
and the frequency determining submodule is used for determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual positions of the specified targets in the target behavior list.
Optionally, the behavior thermodynamic diagram generating device further includes:
the setting instruction acquisition module is used for acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of a sub-area;
And the sub-region setting module is used for determining each sub-region in the region to be counted according to the granularity setting instruction.
Optionally, the sub-region includes the preset monitoring region, and the sub-region frequency statistics module 702 includes:
an inclusion relation determining submodule for obtaining an inclusion relation between each of the sub-areas and each of the preset monitoring areas;
and the behavior frequency counting submodule is used for determining the frequency of the occurrence of the specified behaviors in each sub-area of the area to be counted according to the inclusion relationship, the behavior analysis result of each specified target and the preset monitoring area where each specified target is located.
Optionally, the specific behavior includes a plurality of specific behaviors, and the behavior thermodynamic diagram generating module 703 includes:
the multi-frequency counting submodule is used for acquiring the electronic map of the area to be counted and acquiring the frequency of occurrence of each appointed behavior in each sub-area;
the thermal color corresponding submodule is used for determining the thermal color corresponding to each specified behavior;
and the map coloring submodule is used for displaying the thermal color corresponding to each appointed behavior in the subarea in the map of the subarea according to the occurrence frequency of each appointed behavior in the subarea aiming at any subarea in the electronic map, wherein the depth of any thermal color is positively correlated with the occurrence frequency of the appointed behavior corresponding to the thermal color.
Optionally, the behavior thermodynamic diagram generating device further includes:
and the linkage strategy module is used for executing a linkage strategy of the specified behavior corresponding to the thermal color meeting the preset linkage rule when the thermal color of the specified monitoring area of the behavior thermodynamic diagram meets the preset linkage rule.
Optionally, the behavior analysis result of each specified target is a specified target list for triggering each specified behavior; the analysis result obtaining module includes:
the behavior list receiving submodule is used for receiving a behavior list sent by each intelligent device, wherein the behavior list comprises an identifier of a specified target, and the behavior types of the specified targets in the same behavior list are the same;
and the behavior list assembling submodule is used for assembling each behavior list and respectively obtaining a specified target list for triggering each specified behavior.
An embodiment of the present application further provides a device for sending a behavior list, see fig. 8, which is applied to a front-end intelligent device, and the device includes:
an image data obtaining module 801, configured to obtain image data of a preset monitoring area;
a target behavior analysis module 802, configured to analyze the image data through a computer vision technology to obtain a behavior analysis result of each specified target in the image data;
A designated target classification module 803, configured to classify, according to a behavior analysis result of each designated target, each designated target to obtain multiple behavior lists, where behavior types of each designated target in the same behavior list are the same;
a behavior list sending module 804, configured to send each behavior list.
Optionally, the behavior analysis result of each specified target is a behavior analysis result of each sample specified target, and the target behavior analysis module 802 includes:
a target position determining submodule, configured to determine, through a preset target detection algorithm and a preset target tracking algorithm, each designated target and a position of each designated target in the image data;
the specified target sampling submodule is used for sampling each specified target in the image data through a preset target sampling algorithm to obtain each sampling specified target;
a pixel region intercepting submodule, configured to perform target behavior sequence extraction on the image data according to a position of each of the sampling designated targets, to obtain a pixel region sequence of each of the sampling designated targets;
and the target behavior analysis submodule is used for analyzing the pixel region sequence of each sampling specified target to obtain a behavior analysis result of each sampling specified target.
An embodiment of the present application further provides an electronic device, including: a processor and a memory;
the memory is used for storing computer programs;
the processor is configured to implement any one of the behavior thermodynamic diagrams generating methods described above when executing the computer program stored in the memory.
Optionally, referring to fig. 9, the electronic device according to the embodiment of the present application further includes a communication interface 902 and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904. Specifically, the electronic device may be a server or a hard disk recorder.
An embodiment of the present application further provides an electronic device, including: a processor and a memory;
the memory is used for storing computer programs;
the processor is configured to implement any of the above-described behavior list transmission methods when executing the computer program stored in the memory. Specifically, the electronic device may be a smart camera or a hard disk video recorder, or the like.
An embodiment of the present application further provides an electronic device, including: a processor and a memory;
the memory is used for storing computer programs;
the processor is used for realizing any one of the alarm methods when executing the computer program stored in the memory.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program, when executed by a processor, implements any one of the behavior thermodynamic diagrams generating methods described above.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method implements any of the above-mentioned behavior list sending methods.
An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned alarm methods.
It should be noted that, in this document, the technical features in the various alternatives can be combined to form the scheme as long as the technical features are not contradictory, and the scheme is within the scope of the disclosure of the present application. Relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, and the storage medium, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims (31)

1. A method of alerting, the method comprising:
displaying a behavior thermodynamic diagram of a region to be counted, wherein the behavior thermodynamic diagram represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted;
and when the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition.
2. The method of claim 1, wherein triggering an alarm for a sub-region meeting a preset alarm condition when the sub-region in the behavioral thermodynamic diagram meets the preset alarm condition comprises:
Respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value;
and triggering an alarm aiming at the target sub-region with the frequency of the occurrence of the specified behaviors larger than the preset frequency threshold.
3. The method of claim 1, wherein a thermodynamic color is included in a sub-region of the behavioral thermodynamic diagram, the thermodynamic color characterizing how often a given behavior occurs in the sub-region, and the higher the frequency of occurrence of a given behavior in the sub-region, the higher the thermodynamic value of the thermodynamic color of the sub-region;
when the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition, wherein the alarm comprises the following steps:
respectively comparing the thermal force value of the thermal force color of each subarea with a preset thermal force preset value;
and triggering the alarm aiming at the subarea to be alarmed, wherein the heat value of the subarea to be alarmed is larger than the preset heat threshold value.
4. The method according to claim 1, wherein a sub-region of the behavior thermodynamic diagram comprises a thermal color, the specified behaviors are a plurality of specified behaviors, different specified behaviors correspond to different thermal colors, the depth of the thermal color is positively correlated with the frequency of occurrence of the specified behaviors corresponding to the thermal colors, and each thermal color corresponds to corresponding alarm linkage;
When the sub-region of the behavior thermodynamic diagram meets a preset alarm condition, triggering an alarm aiming at the sub-region meeting the preset alarm condition, wherein the alarm comprises the following steps:
aiming at each thermal color in each sub-area, comparing the depth degree of the thermal color with a preset degree preset value corresponding to the thermal color;
and triggering alarm linkage aiming at the subarea where the target heating power color is located and corresponding to the target heating power color aiming at the target heating power color with the depth degree larger than the preset degree preset value.
5. The method of claim 1, further comprising:
acquiring a display instruction of a user for a sub-region to be displayed;
and displaying the image data in the sub-area to be displayed according to the display instruction, wherein the image data in the sub-area to be displayed is a video stream of a monitoring area in the sub-area to be displayed.
6. A behavior thermodynamic diagram generation method applied to a back-end device, the method comprising:
acquiring a behavior analysis result of each designated target in image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted;
Determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein an intersection exists between the sub-region and the preset monitoring region;
and generating a behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region.
7. The method according to claim 6, wherein the obtaining of the behavior analysis result of each designated target in the image data of each preset monitoring area comprises:
acquiring image data of each preset monitoring area;
determining each designated target and the position of each designated target in each image data through a preset target detection algorithm and a preset target tracking algorithm;
sampling each designated target in each image data through a preset target sampling algorithm to obtain each sampling designated target;
according to the position of each sampling specified target, performing target behavior sequence extraction on each image data to obtain a pixel region sequence of each sampling specified target;
and analyzing the pixel region sequence of each specified target to obtain a behavior analysis result of each specified target.
8. The method according to claim 6, wherein one of the sub-areas comprises at least one of the preset monitoring areas, and the determining the frequency of the occurrence of the specified behavior in each sub-area of the area to be counted according to the behavior analysis result of each specified target comprises:
acquiring the inclusion relation between each sub-area and each preset monitoring area;
and determining the frequency of the designated behaviors in each sub-area of the area to be counted according to the inclusion relation, the behavior analysis result of each designated target and the preset monitoring area where each designated target is located.
9. The method according to claim 6, wherein after the obtaining of the behavior analysis result of each designated target in the image data of each preset monitoring area, the method further comprises:
acquiring the actual position of each designated target in the preset monitoring area;
determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein the determining comprises the following steps:
and determining the occurrence frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target.
10. The method according to claim 9, wherein the determining the frequency of occurrence of the specified behavior in each sub-area of the area to be counted according to the actual position of each specified target and the behavior analysis result of each specified target comprises:
classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
determining a target behavior list corresponding to the specified behavior;
and determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target in the target behavior list.
11. The method according to claim 6, wherein before determining the frequency of occurrence of the specific behavior in each sub-area of the area to be counted according to the behavior analysis result of each specific target, the method further comprises:
acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of a sub-region;
and determining each sub-area in the area to be counted according to the granularity setting instruction.
12. The method according to claim 6, wherein the designated behavior comprises a plurality of designated behaviors, and the generating of the behavior thermodynamic diagram of the designated behavior of the region to be counted according to the frequency of occurrence of the designated behavior in each sub-region comprises:
Acquiring an electronic map of the area to be counted, and acquiring the frequency of each appointed behavior in each sub-area;
determining the thermal color corresponding to each designated behavior;
and aiming at any sub-area in the electronic map, displaying the thermal color corresponding to each specified behavior in the sub-area in the map of the sub-area according to the occurrence frequency of each specified behavior in the sub-area, wherein the shade degree of any thermal color is positively correlated with the occurrence frequency of the specified behavior corresponding to the thermal color.
13. The method of claim 6, wherein the behavior analysis result of each specific target is a specific target list triggering each specific behavior; the acquiring of the behavior analysis result of each designated target in the image data of each preset monitoring area includes:
receiving each behavior list sent by each front-end intelligent device, wherein the behavior lists comprise identifiers of designated targets, and the behavior types of the designated targets in the same behavior list are the same;
and assembling each behavior list to respectively obtain an appointed target list for triggering each appointed behavior.
14. A method for sending a behavior list is applied to a front-end intelligent device, and comprises the following steps:
Acquiring image data of a preset monitoring area;
analyzing the image data through a computer vision technology to obtain a behavior analysis result of each designated target in the image data;
classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
and sending each behavior list.
15. The method of claim 14, wherein the analyzing the behavior of each target is a behavior analysis of each sampled target, and analyzing the image data by computer vision techniques to obtain the behavior analysis of each target in the image data comprises:
determining each designated target and the position of each designated target in the image data through a preset target detection algorithm and a preset target tracking algorithm;
sampling each designated target in the image data through a preset target sampling algorithm to obtain each sampling designated target;
according to the position of each sampling specified target, performing target behavior sequence extraction on the image data to obtain a pixel region sequence of each sampling specified target;
And analyzing the pixel region sequence of each sampling specified target to obtain a behavior analysis result of each sampling specified target.
16. An alarm device, characterized in that the device comprises:
the thermodynamic diagram display module is used for displaying a behavior thermodynamic diagram of the region to be counted, wherein the behavior thermodynamic diagram represents the frequency of occurrence of a specified behavior in each sub-region of the region to be counted;
and the alarm triggering module is used for triggering the alarm aiming at the subarea meeting the preset alarm condition when the subarea of the behavior thermodynamic diagram meets the preset alarm condition.
17. The apparatus of claim 16, wherein the alarm triggering module comprises:
the frequency comparison submodule is used for respectively comparing the occurrence frequency of the specified behaviors in each sub-area of the behavior thermodynamic diagram with the preset frequency threshold value;
and the sub-region alarm sub-module is used for triggering an alarm aiming at the target sub-region with the frequency of the occurrence of the specified behavior larger than the preset frequency threshold.
18. The apparatus of claim 16, wherein a thermodynamic color is included in a sub-region of the behavioral thermodynamic diagram, the thermodynamic color characterizing how often a given behavior occurs in the sub-region, and the higher the frequency of occurrence of a given behavior in the sub-region, the higher the thermodynamic value of the thermodynamic color of the sub-region;
The alarm triggering module comprises:
the thermal force value comparison submodule is used for respectively comparing the thermal force value of the thermal force color of each subarea with the preset thermal force preset value;
and the triggering alarm submodule is used for triggering the alarm aiming at the to-be-alarmed subarea with the heat value larger than the preset thermal threshold value.
19. A behavioral thermodynamic diagram generation apparatus, applied to a backend device, the apparatus comprising:
the analysis result acquisition module is used for acquiring behavior analysis results of each designated target in the image data of each preset monitoring area, wherein the preset monitoring area is an area to be counted;
the sub-region frequency counting module is used for determining the frequency of occurrence of the specified behaviors in each sub-region of the region to be counted according to the behavior analysis result of each specified target, wherein the sub-region and the preset monitoring region have intersection;
and the behavior thermodynamic diagram generating module is used for generating the behavior thermodynamic diagram of the specified behaviors of the region to be counted according to the occurrence frequency of the specified behaviors in each sub-region.
20. The apparatus of claim 19, wherein one of said sub-areas comprises at least one of said predetermined monitoring areas, and wherein said sub-area frequency statistics module comprises:
The inclusion relation determining submodule is used for acquiring the inclusion relation between each sub-area and each preset monitoring area;
and the behavior frequency counting submodule is used for determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the inclusion relationship, the behavior analysis result of each specified target and the preset monitoring area where each specified target is located.
21. The apparatus of claim 19, wherein the sub-region frequency statistics module comprises:
the designated target classification submodule is used for classifying the designated targets according to the behavior analysis result of each designated target to obtain a plurality of behavior lists, wherein the behavior types of the designated targets in the same behavior list are the same;
the target list determining submodule is used for determining a target behavior list corresponding to the specified behavior;
and the frequency determining submodule is used for determining the frequency of the specified behaviors in each sub-area of the area to be counted according to the actual position of each specified target in the target behavior list.
22. The apparatus of claim 19, further comprising:
the setting instruction acquisition module is used for acquiring a granularity setting instruction input by a user, wherein the granularity setting instruction represents the size attribute of a sub-region;
And the sub-region setting module is used for determining each sub-region in the region to be counted according to the granularity setting instruction.
23. The apparatus of claim 19, wherein the designated behavior is a plurality of designated behaviors, wherein the behavior thermodynamic diagram generation module comprises:
the multi-frequency counting submodule is used for acquiring the electronic map of the area to be counted and acquiring the frequency of each appointed behavior in each sub-area;
the thermal color corresponding submodule is used for determining the thermal color corresponding to each specified behavior;
and the map coloring submodule is used for displaying the thermal color corresponding to each appointed behavior in the sub-area in the map of the sub-area according to the frequency of the appointed behavior in the sub-area aiming at any sub-area in the electronic map, wherein the depth of any thermal color is positively correlated with the frequency of the appointed behavior corresponding to the thermal color.
24. The apparatus of claim 19, wherein the behavior analysis result of each specific target is a specific target list triggering each specific behavior; the analysis result acquisition module comprises:
the behavior list receiving submodule is used for receiving a behavior list sent by each intelligent device, wherein the behavior list comprises an identifier of a specified target, and the behavior types of the specified targets in the same behavior list are the same;
And the behavior list assembling submodule is used for assembling each behavior list and respectively obtaining a specified target list for triggering each specified behavior.
25. An apparatus for sending a behavior list, applied to a front-end smart device, the apparatus comprising:
the image data acquisition module is used for acquiring image data of a preset monitoring area;
the target behavior analysis module is used for analyzing the image data through a computer vision technology to obtain a behavior analysis result of each designated target in the image data;
the specified target classification module is used for classifying the specified targets according to the behavior analysis result of each specified target to obtain a plurality of behavior lists, wherein the behavior types of the specified targets in the same behavior list are the same;
and the behavior list sending module is used for sending each behavior list.
26. An electronic device comprising a processor and a memory;
the memory is used for storing a computer program;
the processor, when executing the program stored in the memory, implements the alarm method of any one of claims 1 to 5.
27. An electronic device comprising a processor and a memory;
The memory is used for storing a computer program;
the processor is configured to implement the behavior thermodynamic diagram generation method according to any one of claims 6 to 13 when executing the program stored in the memory.
28. An electronic device comprising a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to implement the behavior list transmission method according to any one of claims 14 to 15 when executing the program stored in the memory.
29. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the behavior thermodynamic diagram generation method according to any one of claims 6 to 13.
30. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the behavior list transmission method according to any one of claims 14 to 15.
31. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the alarm method of any one of claims 1 to 5.
CN201910351634.4A 2019-04-28 2019-04-28 Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium Active CN111862521B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201910351634.4A CN111862521B (en) 2019-04-28 2019-04-28 Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium
PCT/CN2020/085423 WO2020221031A1 (en) 2019-04-28 2020-04-17 Behavior thermodynamic diagram generation and alarm method and apparatus, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910351634.4A CN111862521B (en) 2019-04-28 2019-04-28 Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111862521A true CN111862521A (en) 2020-10-30
CN111862521B CN111862521B (en) 2022-07-05

Family

ID=72965214

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910351634.4A Active CN111862521B (en) 2019-04-28 2019-04-28 Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium

Country Status (2)

Country Link
CN (1) CN111862521B (en)
WO (1) WO2020221031A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112488913A (en) * 2020-11-27 2021-03-12 杭州海康威视数字技术股份有限公司 Data processing method and device and electronic equipment
CN113010829A (en) * 2021-03-31 2021-06-22 建信金融科技有限责任公司 Data partition visualization method and device, computer equipment and storage medium
CN113592320A (en) * 2021-06-16 2021-11-02 成都世纪光合作用科技有限公司 Dining intention identification method and device and electronic equipment
CN115083112A (en) * 2022-08-22 2022-09-20 枫树谷(成都)科技有限责任公司 Intelligent early warning emergency management system and deployment method thereof
CN115648630A (en) * 2022-10-25 2023-01-31 上海复志信息科技股份有限公司 Processing method and device for release film printing area and photocuring 3D printing equipment

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112560715A (en) * 2020-12-21 2021-03-26 北京市商汤科技开发有限公司 Operation record display method and device, electronic equipment and storage medium
CN112749461B (en) * 2020-12-25 2023-05-02 深圳供电局有限公司 Load data monitoring method, power system, computer device and storage medium
CN112906594B (en) * 2021-03-03 2022-06-03 杭州海康威视数字技术股份有限公司 Defense deployment area generation method, device, equipment and storage medium
CN114474091B (en) * 2022-01-26 2024-02-27 北京声智科技有限公司 Robot killing method, killing robot, killing device and storage medium
CN114579889B (en) * 2022-04-26 2022-08-12 阿里巴巴（中国）有限公司 Recommendation method, device, equipment and storage medium for order thermodynamic diagrams
CN115099620B (en) * 2022-06-23 2023-12-05 中国建筑第五工程局有限公司 BIM-based intelligent building construction information collection and analysis system

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104766295A (en) * 2014-01-02 2015-07-08 三星泰科威株式会社 Heatmap providing apparatus and method
CN105447458A (en) * 2015-11-17 2016-03-30 深圳市商汤科技有限公司 Large scale crowd video analysis system and method thereof
WO2016135536A1 (en) * 2015-02-27 2016-09-01 Yandex Europe Ag Method of and system for generating a heat map
CN106251578A (en) * 2016-08-19 2016-12-21 深圳奇迹智慧网络有限公司 Artificial abortion's early warning analysis method and system based on probe
CN106485868A (en) * 2015-08-27 2017-03-08 杭州海康威视数字技术股份有限公司 The monitoring server of the monitoring method of the condition of a fire, system and the condition of a fire
US20170213436A1 (en) * 2016-01-26 2017-07-27 Flir Systems, Inc. Systems and methods for behavioral based alarms
CN107292271A (en) * 2017-06-23 2017-10-24 北京易真学思教育科技有限公司 Learning-memory behavior method, device and electronic equipment
CN108038116A (en) * 2017-10-24 2018-05-15 安徽四创电子股份有限公司 A kind of density of stream of people monitoring method based on GIS
CN108229407A (en) * 2018-01-11 2018-06-29 武汉米人科技有限公司 A kind of behavioral value method and system in video analysis
CN108399591A (en) * 2018-02-12 2018-08-14 北京天时前程自动化工程技术有限公司 Heat supply network data visualization monitoring method and system of the thermal substation for thermal region
CN108419045A (en) * 2018-02-11 2018-08-17 浙江大华技术股份有限公司 A kind of monitoring method and device based on infrared thermal imaging technique
CN108428326A (en) * 2018-03-14 2018-08-21 海南师范大学 A kind of eco-tour assessment management precaution information system
CN108764047A (en) * 2018-04-27 2018-11-06 深圳市商汤科技有限公司 Group's emotion-directed behavior analysis method and device, electronic equipment, medium, product
CN108846389A (en) * 2018-08-13 2018-11-20 树蛙信息科技(南京)有限公司 A kind of passenger flow analysis system and its method
KR20180130621A (en) * 2017-05-29 2018-12-10 전자부품연구원 Apparatus and Method for Analyzing Overt Behavioral Response in Service Space
CN109034355A (en) * 2018-07-02 2018-12-18 百度在线网络技术（北京）有限公司 Number prediction technique, device, equipment and the storage medium of fine and close crowd
CN109508657A (en) * 2018-10-29 2019-03-22 重庆中科云丛科技有限公司 Crowd massing analysis method, system, computer readable storage medium and equipment
CN109635769A (en) * 2018-12-20 2019-04-16 天津天地伟业信息系统集成有限公司 A kind of Activity recognition statistical method for ball-shaped camera

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9997046B2 (en) * 2015-12-31 2018-06-12 International Business Machines Corporation Visitor flow management

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104766295A (en) * 2014-01-02 2015-07-08 三星泰科威株式会社 Heatmap providing apparatus and method
WO2016135536A1 (en) * 2015-02-27 2016-09-01 Yandex Europe Ag Method of and system for generating a heat map
CN106485868A (en) * 2015-08-27 2017-03-08 杭州海康威视数字技术股份有限公司 The monitoring server of the monitoring method of the condition of a fire, system and the condition of a fire
CN105447458A (en) * 2015-11-17 2016-03-30 深圳市商汤科技有限公司 Large scale crowd video analysis system and method thereof
US20170213436A1 (en) * 2016-01-26 2017-07-27 Flir Systems, Inc. Systems and methods for behavioral based alarms
CN106251578A (en) * 2016-08-19 2016-12-21 深圳奇迹智慧网络有限公司 Artificial abortion's early warning analysis method and system based on probe
KR20180130621A (en) * 2017-05-29 2018-12-10 전자부품연구원 Apparatus and Method for Analyzing Overt Behavioral Response in Service Space
CN107292271A (en) * 2017-06-23 2017-10-24 北京易真学思教育科技有限公司 Learning-memory behavior method, device and electronic equipment
CN108038116A (en) * 2017-10-24 2018-05-15 安徽四创电子股份有限公司 A kind of density of stream of people monitoring method based on GIS
CN108229407A (en) * 2018-01-11 2018-06-29 武汉米人科技有限公司 A kind of behavioral value method and system in video analysis
CN108419045A (en) * 2018-02-11 2018-08-17 浙江大华技术股份有限公司 A kind of monitoring method and device based on infrared thermal imaging technique
CN108399591A (en) * 2018-02-12 2018-08-14 北京天时前程自动化工程技术有限公司 Heat supply network data visualization monitoring method and system of the thermal substation for thermal region
CN108428326A (en) * 2018-03-14 2018-08-21 海南师范大学 A kind of eco-tour assessment management precaution information system
CN108764047A (en) * 2018-04-27 2018-11-06 深圳市商汤科技有限公司 Group's emotion-directed behavior analysis method and device, electronic equipment, medium, product
CN109034355A (en) * 2018-07-02 2018-12-18 百度在线网络技术（北京）有限公司 Number prediction technique, device, equipment and the storage medium of fine and close crowd
CN108846389A (en) * 2018-08-13 2018-11-20 树蛙信息科技(南京)有限公司 A kind of passenger flow analysis system and its method
CN109508657A (en) * 2018-10-29 2019-03-22 重庆中科云丛科技有限公司 Crowd massing analysis method, system, computer readable storage medium and equipment
CN109635769A (en) * 2018-12-20 2019-04-16 天津天地伟业信息系统集成有限公司 A kind of Activity recognition statistical method for ball-shaped camera

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112488913A (en) * 2020-11-27 2021-03-12 杭州海康威视数字技术股份有限公司 Data processing method and device and electronic equipment
CN113010829A (en) * 2021-03-31 2021-06-22 建信金融科技有限责任公司 Data partition visualization method and device, computer equipment and storage medium
CN113592320A (en) * 2021-06-16 2021-11-02 成都世纪光合作用科技有限公司 Dining intention identification method and device and electronic equipment
CN113592320B (en) * 2021-06-16 2023-10-03 成都世纪光合作用科技有限公司 Dining intention recognition method and device and electronic equipment
CN115083112A (en) * 2022-08-22 2022-09-20 枫树谷(成都)科技有限责任公司 Intelligent early warning emergency management system and deployment method thereof
CN115083112B (en) * 2022-08-22 2022-11-22 枫树谷(成都)科技有限责任公司 Intelligent early warning emergency management system and deployment method thereof
CN115648630A (en) * 2022-10-25 2023-01-31 上海复志信息科技股份有限公司 Processing method and device for release film printing area and photocuring 3D printing equipment

Also Published As

Publication number Publication date
CN111862521B (en) 2022-07-05
WO2020221031A1 (en) 2020-11-05

Similar Documents

Publication Publication Date Title
CN111862521B (en) Behavior thermodynamic diagram generation and alarm method and device, electronic equipment and storage medium
US10614316B2 (en) Anomalous event retriever
US20240037953A1 (en) Methods and systems for determining object activity within a region of interest
US9792434B1 (en) Systems and methods for security data analysis and display
US10318910B2 (en) Spatio-temporal key performance indicators
AU2019100806A4 (en) Real-Time Crowd Measurement And Management Systems And Methods Thereof
US20160065906A1 (en) Video Analytics Configuration
Wang et al. Tweeting cameras for event detection
Khan et al. Unmanned aerial vehicle-based traffic analysis: A case study to analyze traffic streams at urban roundabouts
US11575837B2 (en) Method, apparatus and computer program for generating and displaying a heatmap based on video surveillance data
CN111222373B (en) Personnel behavior analysis method and device and electronic equipment
US20140355823A1 (en) Video search apparatus and method
US20220044558A1 (en) Method and device for generating a digital representation of traffic on a road
CN111582060B (en) Automatic line drawing perimeter alarm method, computer equipment and storage device
CN111369792B (en) Traffic incident analysis method and device and electronic equipment
CN113920585A (en) Behavior recognition method and device, equipment and storage medium
EP3940666A1 (en) Digital reconstruction method, apparatus, and system for traffic road
Dobler et al. Patterns of urban foot traffic dynamics
CN115797125B (en) Rural digital intelligent service platform
CN110855932B (en) Alarm method and device based on video data, electronic equipment and storage medium
Sinnott et al. Estimating micro-populations through social media analytics
GB2594459A (en) A method, apparatus and computer program for generating and displaying a heatmap based on video surveillance data
CN111291597B (en) Crowd situation analysis method, device, equipment and system based on image
CN112668364A (en) Behavior prediction method and device based on video
Li et al. Region-based trajectory analysis for abnormal behaviour detection: a trial study for suicide detection and prevention

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant