CN110674761B - Regional behavior early warning method and system - Google Patents
Regional behavior early warning method and system Download PDFInfo
- Publication number
- CN110674761B CN110674761B CN201910921181.4A CN201910921181A CN110674761B CN 110674761 B CN110674761 B CN 110674761B CN 201910921181 A CN201910921181 A CN 201910921181A CN 110674761 B CN110674761 B CN 110674761B
- Authority
- CN
- China
- Prior art keywords
- information
- person
- future
- scene
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000006399 behavior Effects 0.000 claims abstract description 72
- 230000000694 effects Effects 0.000 claims abstract description 60
- 238000013528 artificial neural network Methods 0.000 claims abstract description 40
- 230000033001 locomotion Effects 0.000 claims abstract description 40
- 230000015654 memory Effects 0.000 claims abstract description 29
- 238000012544 monitoring process Methods 0.000 claims description 23
- 230000003993 interaction Effects 0.000 claims description 18
- 230000000007 visual effect Effects 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 230000003542 behavioural effect Effects 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 description 18
- 230000009471 action Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 11
- 206010000117 Abnormal behaviour Diseases 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 230000001815 facial effect Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 241001622623 Coeliadinae Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Tourism & Hospitality (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Security & Cryptography (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Alarm Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a regional behavior early warning method and a system, wherein the embodiment of the invention inputs figure image information acquired in a region into a set first neural network, and outputs and obtains scene information, figure limb information, relationship information between figures and the scene and relationship information between the figures and objects in the scene; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the person, and the second neural network outputs the future activity tag and the future target position information of the person; and judging whether the character behavior is suspicious according to the output future motion track information of the character, the future activity label and the future purpose information of the character. The embodiment of the invention adopts a plurality of neural networks to accurately predict the information related to the future activities of the people so as to judge whether the behavior of the people is suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a regional behavior early warning method and a regional behavior early warning system.
Background
With the great progress of algorithm optimization and prediction accuracy of the deep neural network in recent years, the deep learning can be applied to all the fields of life to almost obtain good effect, and a large amount of labor cost is saved. With the increasing of urban residents in recent years, the regional security early warning faces unprecedented challenges, and good security effect is difficult to obtain by means of old-fashioned monitoring and artificial checking.
In order to improve the effect of the regional behavior early warning, a deep neural network can be introduced for realization, and specifically, the following modes are available for realization.
The patent application with the publication number of CN109064698A discloses a resident security early warning system, which identifies whether a person entering a cell is a resident of a non-local cell or not through a face; for the residents in the non-local community, continuously acquiring the path track and the behavior pattern of the residents in the non-local community in the community, wherein the path track and the behavior pattern at least comprise the behavior determination of the residents in and out of the non-local community; accumulating the total number of units in the residential area of the non-local residential area in a set time period, and acquiring the residence time of each unit in the residential area of the non-local residential area; when the total number of units of residents in the non-local community entering and exiting the community within a set time period is not less than N, judging whether the residents in the non-local community have abnormal behaviors or not according to the residence time of the residents in each unit of the non-local community, and determining the grade of the abnormal behaviors, wherein N is an integer and is not less than 3; and sending out an alarm according to different abnormal behavior levels.
The patent application with publication number CN109189078A discloses a household safety protection robot and a method based on deep reinforcement learning, which share the behavior state information of a target person, the position of the target person and the position of an obstacle in the environment through deep learning, and output the moving track of the target person, thereby realizing locking tracking of the target person.
Patent application publication No. CN105975633A discloses a method and apparatus for obtaining motion trajectory. Predicting the motion track of a preset figure according to the current geographic position and the motion state of the preset figure corresponding to the preset figure information; and starting the target shooting equipment on the motion trail.
Patent application with publication number CN108877121A discloses an artificial intelligence early warning system based on cloud platform, mainly through the face collection module who locates the district entrance for gather the face of discrepancy district, and form face identification information and send cloud ware, compare through the photo with criminal, form warning signal.
It can be seen that, when the above scheme is used for realizing the regional behavior early warning, the region is a cell as an example, and a face recognition unit is mostly arranged at a position such as a cell entrance guard, so that the identity of a person entering the cell is determined through face recognition, or the identity is compared with a photo of a criminal through a cloud platform, and the early warning is realized. However, once a relevant person enters a cell, the track of the relevant person can be recorded only through the set traditional monitoring camera, and abnormal behaviors are judged through artificial inspection or a simple statistical method to give an early warning. The manual inspection of video monitoring consumes a large amount of labor cost, continuous monitoring cannot be achieved when personnel change posts and the spirit is not concentrated, the inspection of video monitoring by adopting a statistical method is not accurate, and the statistical standard value is difficult to determine. No matter which mode is adopted, the early warning accuracy rate is low.
Therefore, how to realize the regional behavior early warning on the basis of improving the early warning accuracy and improving the early warning effect is a problem to be solved urgently.
Disclosure of Invention
In view of this, the embodiment of the present invention provides a regional behavior early warning method, which can improve early warning accuracy and improve early warning effect.
The embodiment of the invention also provides a regional behavior early warning system, which can improve the early warning accuracy and improve the early warning effect.
The embodiment of the invention is realized as follows:
a regional behavior early warning method comprises the following steps:
inputting the image information of the person collected in the area into a first set neural network, and outputting the information of the scene in the area, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene;
abstracting scene information, figure limb information, relationship information between a figure and a scene, and relationship information between the figure and an object in the scene in the area into an information tensor;
inputting the abstract information tensor into a set memory network model and a set second neural network, so that the memory network model outputs the future motion trail information of the person, and the second neural network outputs the future activity label and the future target position information of the person;
and judging whether the character behavior is suspicious according to the output future motion track information of the character, the future activity label and the future purpose information of the character.
Before the acquiring of the person image information in the region, the method further comprises:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
The determining whether the person behavior is suspicious further comprises:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future purpose information and the identity information of the person.
The abstract information tensor is input into a set memory network model, and the information of the future motion trail of the figure is output as follows:
and performing compression coding on the abstract information tensor by using a memory network model to obtain a visual characteristic tensor, decoding by using the memory network model, and outputting characteristic coordinate information of a future motion trail of a person on an actual coordinate plane.
The first neural network is updated in real time; updating the memory network model in real time; the second neural network is updated in real time.
The method further comprises the following steps:
and if the person behavior is judged to be suspicious, early warning is carried out.
A regional behavioral early warning system, comprising: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first neural network, setting a memory network model and a second neural network;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network, outputting and obtaining scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor into a set memory network model so as to enable the memory network model to output the information of the future motion track of the person;
the activity prediction module is used for inputting the abstract information tensor into a set second neural network, and the second neural network outputs future activity labels and future target position information of the person;
and the abnormity alarm module is used for judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
The system further comprises:
and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
The track generation module is further used for carrying out compression coding on the abstract information tensor through a memory network model, decoding the abstract information tensor through the memory network model after the visual characteristic tensor is obtained, and outputting characteristic coordinate information of the future motion track of the person on the actual coordinate plane.
And the abnormity alarm module is used for early warning when the person behavior is judged to be suspicious.
As can be seen from the above, the embodiment of the present invention inputs the person image information collected in the area into the set first neural network, and outputs the obtained scene information, the person limb information, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene in the area; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the figure, and the second neural network outputs the future activity tag and the future target position information of the figure; and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person. The embodiment of the invention adopts a plurality of neural networks to accurately predict the relevant information of the future activities of the people so as to judge whether the behaviors of the people are suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Drawings
Fig. 1 is a flowchart of a regional behavior early warning method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a regional behavior early warning system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an execution process of a face and identity recognition module according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a processing procedure of a character behavior module according to an embodiment of the present invention;
FIG. 5 is a schematic processing flow diagram of a character interaction module according to an embodiment of the present invention;
FIG. 6 is a block diagram of an activity prediction module according to an embodiment of the present invention;
fig. 7 is a schematic processing flow diagram of an exception alarm module according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples.
It can be seen from the background art that the main reasons for the low accuracy and poor warning effect of the warning during the regional behavior warning are that the warning is implemented only by using the face recognition unit to determine suspicious people, or using the surveillance camera technology to track the trace of people, and manually or statistically determining whether the behavior of people is suspicious. Furthermore, most behavior predictions about people described in the background art are trajectory predictions, and the main purpose is to predict the next traveling direction of a suspicious person, so as to perform real-time tracking, which does not involve future behavior trajectory prediction of people and has low early warning accuracy.
With the development of Artificial Intelligence (AI) technology, AI technology is widely used. Along with the continuous proposition of more advanced network models and algorithms, the accuracy of AI application is improved, and the artificial inspection can be replaced in various fields in the future. Therefore, in order to overcome the problems generated in the regional behavior early warning in the background technology, the embodiment of the invention applies the most advanced action track and behavior prediction of video characters to the regional behavior early warning to replace the manual monitoring and statistical method in the background technology, not only can determine the identity of the characters, but also can train a plurality of set deep neural networks through a large amount of training data, predict the abnormal behaviors of the characters in real time through the plurality of deep neural networks, achieve early warning and prevent the diseases in the bud. Specifically, the embodiment of the invention inputs the image information of the person collected in the area into the set first neural network, and outputs the information to obtain the scene information, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene; abstracting the obtained information into an information tensor, and respectively inputting the information tensor into a set memory network model and a set second neural network, wherein the memory network model outputs the future motion trail information of the figure, and the second neural network outputs the future activity tag and the future target position information of the figure; and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
Here, the memory network model is a Long Short Term Memory (LSTM) network, and the neural network employs CNN, which is described in detail below as an example.
Therefore, the embodiment of the invention adopts a plurality of neural networks to accurately predict the information related to the future activities of the person so as to judge whether the behavior of the person is suspicious, thereby improving the early warning accuracy and improving the early warning effect.
Fig. 1 is a flowchart of a regional behavior early warning method provided in an embodiment of the present invention, which specifically includes the steps of:
step 101, inputting the image information of the person collected in the area into a set first CNN, and outputting the information of the scene in the area, the information of the limbs of the person, the information of the relationship between the person and the scene, and the information of the relationship between the person and the object in the scene;
103, inputting the abstract information tensor into a set LSTM network and a set second CNN, so that the LSTM network outputs the future motion track information of the person, and the second CNN outputs the future activity label and the future target position information of the person;
and step 104, judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
In the method, before the capturing of the person image information in the region, the method further includes:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
In this case, the determining whether the person behavior is suspicious further includes:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future destination information and the identity information of the person.
In the method, the abstract information tensor is input into a set LSTM network, and the information of the future motion trail of the person is output and obtained as follows:
and performing compression coding on the abstract information tensor through an LSTM network to obtain a visual characteristic tensor, decoding by adopting the LSTM network, and outputting characteristic coordinate information of a future motion track of a person on an actual coordinate plane.
In the method, the first CNN is updated in real time; the LSTM is updated in real time; the second CNN is updated in real time. Therefore, the deep neural networks of the embodiment of the invention are actually a network capable of continuously learning and optimizing, and the recorded information can be used for the training process of the deep neural networks, so that the accuracy of network judgment is improved.
In this method, the method further comprises:
if the person behavior is judged to be suspicious, early warning is carried out
Fig. 2 is a schematic structural diagram of a regional behavior early warning system provided in an embodiment of the present invention, including: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first CNN, and setting an LSTM and a second CNN;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network CNN, outputting and obtaining scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor to a set LSTM network so as to enable the LSTM network to output information of the future motion track of the person;
the activity prediction module is used for inputting the abstract information tensor into a set second CNN, and the second CNN outputs future activity labels and future target position information of the person;
and the abnormity alarm module is used for judging whether the character behavior is suspicious according to the output future motion trail information of the character, the future activity label of the character and the future purpose information.
The system further comprises: and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
In the system, the track generation module is further configured to perform compression coding on the abstract information tensor through an LSTM network, obtain a visual feature tensor, perform decoding through the LSTM network, and output feature coordinate information of a future motion track of a person on an actual coordinate plane.
In the system, the first CNN is updated in real time; the LSTM is updated in real time; the second CNN is updated in real time.
In the system, the abnormity warning module is used for carrying out early warning when the person behavior is judged to be suspicious.
Fig. 3 is a schematic diagram of an execution process of the face and identity recognition module according to the embodiment of the present invention. The face and identity recognition module is generally arranged in an area, such as a main passage and an entrance/exit of a large-scale activity place, and mainly comprises a camera shooting unit, a face acquisition unit, a facial feature extraction unit, a face matching unit and an identity confirmation unit. Specifically, as shown in fig. 3, wherein:
and 305, the identity confirmation unit confirms whether the person is the person inside or outside the area according to the matching result.
In the embodiment of the present invention, the character behavior module is configured to set the first CNN, set the LSTM, and the second CNN, and when the first CNN, the LSTM, and the second CNN are set, the visual information of each person appearing in the monitoring camera scene is encoded, and in this process, the shape and the body movement of the character are modeled instead of abstracting the character into one point, as shown in fig. 4, where fig. 4 is a schematic processing process diagram of the character behavior module provided in the embodiment of the present invention. In order to model the outline of a person in a regional scene, adopting an object detection model with RolAlign to extract the CNN characteristic of the set size of each person bounding box; to obtain a physical motion model of a person, a microsoft image recognition (MSCOCO) network trained on a microsoft image recognition dataset was used to extract person motion keypoint information.
In the embodiment of the invention, the character interaction module is responsible for viewing the interaction between the character and the surrounding environment, including the interaction between the character and the scene and the interaction between the character and the object in the scene. The interaction between the person and the scene is to identify the scene near the person. As shown in fig. 5, fig. 5 is a schematic processing flow diagram of a character interaction module according to an embodiment of the present invention. Firstly, deriving pixel level scene semantic classification of each frame by using a trained scene segmentation model, and dividing parts such as roads, sidewalks and the like in a scene; and then selecting the set size to determine the environment area needing to be identified by the model. For example, the set size is 3, the range of 3 × 3 around the person is selected as the observation region, the information obtained at different times is input to the LSTM, the relationship information between the person and the scene is finally obtained, and then the scene information, the person limb information, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene in the region are abstracted into one information tensor.
In the embodiment of the invention, the track generation module inputs the abstract information tensor into a set LSTM network, so that the LSTM network outputs the future motion track information of the person, the information is compressed into a visual feature tensor Q by an LSTM network encoder, and then the future motion track information of the person is obtained by decryption by a decryptor of the LSTM network.
In the embodiment of the invention, the activity prediction module has two tasks and determines the type and the occurrence place of the future activity of the person. Accordingly, it includes two parts, activity location prediction and activity tag prediction of the manhattan mesh. The role of activity label prediction is to guess what the final purpose of the character in the picture is, and predict the activity at some instant in the future. The active tags are not limited to one at a time, such as a person may walk and carry an item at the same time. The function of activity location prediction is to correct errors for the trajectory generation module, which determines the final destination of the character to compensate for the deviation between the trajectory generation module and the activity tag prediction. The method comprises two tasks of position classification and position regression. The purpose of the location classification is to predict a grid block where the final location coordinate is located, as shown in fig. 6, fig. 6 is a schematic structural diagram of an activity prediction module provided in the embodiment of the present invention. The goal of positional regression is to predict the deviation of the grid block center (dots in the figure) from the final positional coordinates (end of arrow).
The abnormal alarm module in the embodiment of the present invention may analyze whether the behavior of the person is suspicious according to the results of the face and identity recognition module and the activity prediction module, so as to early warn the suspicious behavior of the suspicious person, as shown in fig. 7, fig. 7 is a schematic processing flow diagram of the abnormal alarm module provided in the embodiment of the present invention, and the specific steps are:
step 701, obtaining personnel identity information from a face and identity recognition module;
step 704, determining whether the behavior of the person is abnormal, if so, executing step 705; otherwise, returning to step 703 to continue execution;
By adopting the embodiment of the invention, the monitored personnel comprise personnel in the area and external personnel. For the cell environment, the method comprises the steps of owner, property personnel, external personnel and the like; for villa environments, including owners, visitors, maintenance workers, security guards, unknown personnel and the like; the warehouse environment comprises custodians, pickup personnel, delivery personnel, external personnel and the like; for the bank environment, the bank environment comprises staff, business handling personnel, security personnel, suspicious personnel and the like; the market environment comprises guests, shop assistants, cleaning personnel, security guards and the like.
By adopting the embodiment of the invention, the predicted character future activity information can comprise: external personnel enter the corridor along with a owner, stand by to observe the work and rest rule of residents in the community, check the structure of the building, monitor the camera position and the passage of the community; burglary and destruction behaviors such as turning into a courtyard wall, prying open a door lock and the like; illegal entry, theft of confidential materials, technology, documents, etc.; the theft and robbery actions are carried out in banks and malls and are not limited to the above suspicious actions.
The notification objects for the criminal behavior early warning by adopting the embodiment of the invention comprise security personnel, policemen and owners and are not limited to the personnel. The real-time monitoring and early warning method includes common early warning methods such as short message early warning, telephone early warning, alarm bell early warning, internet of things (IOT) device early warning, video early warning and the like, but is not limited to the above early warning methods.
By adopting the embodiment of the invention, the area comprises but is not limited to a community environment, a villa environment, a storehouse environment, a bank environment and a shopping mall environment. The concept of the area is also applicable to environments which are complicated for other people and are easy to steal and robbery.
The following description will be made by taking several specific examples
The first embodiment is as follows: the area is designated as a cell. By adopting the embodiment of the invention, in a community environment, early warning is given to the fact that outside personnel enter a corridor along with a owner in advance, the residence is sufficient to observe the work and rest rules of residents in the community, and abnormal behaviors such as building structures, monitoring camera positions, community access channels and the like are checked.
The specific process is as follows:
the first step is that the entrance or the main trunk of the cell detects that external people enter the cell through a face and identity recognition module.
And a second step of predicting future action paths and behaviors of external people through the person behavior module, the person interaction module, the track generation module and the activity prediction module according to video monitoring.
And thirdly, the abnormity alarm module finds that the abnormity alarm module follows the owner to move to the corridor according to the information, holds the residence to observe the work and rest rules of residents in the community, or checks abnormal behaviors such as building structures, monitoring camera positions and community access channels. And alarming in advance to remind security personnel to closely observe subsequent actions of external personnel. And when necessary, the person goes out to ask for a question.
The second embodiment: the area is determined as a villa, and early warning is carried out for theft and destruction behaviors such as turning into a yard wall, prying open a door lock and the like outside the single villa by adopting the embodiment of the invention.
The specific process is as follows:
the first step is that a camera is arranged on a high-position wall of the single villa, and whether people outside the villa yard is a master or not is judged through face recognition.
And secondly, predicting the future action path and the future action of the person through the person behavior module, the person interaction module, the track generation module and the activity prediction module according to the monitoring of the video.
And in the third step, if abnormal behaviors are found by the abnormal alarm module according to the information, the APP at the mobile phone end can give an alarm to the owner and security personnel in advance, and the automatic playing and video recording function is started. And the IOT sound equipment in the villa can be started at the same time to play the warning voice.
Example three: the area is set to be in a factory storehouse, and the embodiment of the invention is adopted to carry out early warning on illegal entry and theft of confidential materials, technologies, files and the like.
The specific process is as follows:
the first step is that suspicious people are detected to enter through face recognition at an entrance or a main trunk of a storehouse, and the identity of the suspicious people is judged firstly.
And secondly, predicting the future action path and the future action of the person through the person behavior module, the person interaction module, the track generation module and the activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module gives an alarm in advance according to the information, such as the suspicion that confidential materials, technologies and files are stolen, so as to remind security personnel to closely observe the condition of the camera at the position, and the entrance and the exit of the storehouse are blocked to implement capture.
Example four: the region is set in a bank, and the embodiment of the invention can be used for early warning the actions of robbery and theft to be implemented, reminding security personnel to pay attention and alarming to public security if necessary.
The specific process is as follows:
the first step is that each person entering the bank is detected through face recognition at the entrance and the main corner of the bank, the identity of the person is judged firstly, and if the person is the escaped person recorded on a case, an alarm is triggered immediately.
And secondly, predicting the future action path and the future action of the person through a person behavior module, a person interaction module, a track generation module and an activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module is communicated with wireless equipment on the security personnel in advance according to the information if the suspicion of robbery and theft is found, and the security personnel is reminded to closely observe the movement of the security personnel. And directly informing the police to give an alarm if necessary according to different security levels.
Example five: the area is set in a large-scale shopping mall, and the embodiment of the invention is adopted to early warn the robbery and the theft to be implemented, remind security personnel to pay attention and alarm to public security if necessary.
The specific process is as follows:
the method comprises the following steps that firstly, everyone entering a large shopping mall is detected through face recognition at an entrance and a main corner of the large shopping mall, the identity of the person is judged, and if the person is a person who runs away and is recorded on a case, an alarm is triggered immediately.
And secondly, predicting the future action path and the future action of the person through the person behavior module, the person interaction module, the track generation module and the activity prediction module according to the monitoring of the video.
And thirdly, the abnormity alarm module is communicated with wireless equipment on the security personnel in advance according to the information if the suspicion of robbery and theft is found, and the security personnel is reminded to track and observe the personnel.
And step four, if the person implements a criminal behavior in the early warning process, the person can play the track of the person in real time through a large screen of a networked mall, and security guards at all places of the mall are reminded to block.
Therefore, the embodiment of the invention provides intelligent monitoring in the area, not only provides more advanced and effective future behavior prediction and early warning for suspicious personnel, but also reduces the heavy workload of security personnel in the area to check a plurality of monitoring videos.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A regional behavior early warning method is characterized by comprising the following steps:
inputting the image information of the person collected in the area into a first set neural network, and outputting the information of the scene in the area, the limb information of the person, the relationship information between the person and the scene, and the relationship information between the person and the object in the scene;
abstracting scene information, figure limb information, relationship information between a figure and a scene, and relationship information between the figure and an object in the scene in the area into an information tensor;
inputting the abstract information tensor into a set memory network model and a set second neural network so that the memory network model outputs the future motion trail information of the person, and the second neural network outputs the future activity label and the future target position information of the person;
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
2. The method of claim 1, wherein prior to capturing the image information of the person within the region, the method further comprises:
and identifying the person appearing in the set monitoring camera by adopting a face and identity identification mode, and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
3. The method of claim 2, wherein determining whether the person behavior is suspicious further comprises:
and judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label of the person, the future purpose information and the identity information of the person.
4. The method of claim 1, wherein the abstract information tensor is input into a set memory network model, and the future motion trail information of the person is output as follows:
and performing compression coding on the abstract information tensor through a memory network model to obtain a visual characteristic tensor, decoding by using the memory network model, and outputting characteristic coordinate information of a future motion trail of a person on an actual coordinate plane.
5. The method of claim 1, in which the first neural network is updated in real-time; updating the memory network model in real time; the second neural network is updated in real time.
6. The method of claim 1, wherein the method further comprises:
and if the person behavior is judged to be suspicious, early warning is carried out.
7. A regional behavioral early warning system, comprising: a character behavior module, a character interaction module, a track generation module and an activity prediction module, wherein,
the character behavior module is used for setting a first neural network, a memory network model and a second neural network;
the character interaction module is used for inputting character image information acquired in the region into a set first neural network, outputting and obtaining scene information, character limb information, relationship information between characters and the scene and relationship information between the characters and objects in the scene in the region, and abstracting the scene information, the character limb information, the relationship information between the characters and the scene and the relationship information between the characters and the objects in the scene into an information tensor;
the track generation module is used for inputting the abstract information tensor into a set memory network model so as to enable the memory network model to output the information of the future motion track of the person;
an activity prediction module for inputting the abstracted information tensor into a second neural network, the second neural network outputting future activity labels and future destination location information of the person;
and the abnormity alarm module is used for judging whether the behavior of the person is suspicious according to the output future motion track information of the person, the future activity label and the future purpose information of the person.
8. The system of claim 7, wherein the system further comprises:
and the face and identity recognition module is used for recognizing the person appearing in the set monitoring camera and judging the identity information of the person, wherein the identity information comprises internal personnel or external personnel.
9. The system of claim 7, wherein the trajectory generation module is further configured to perform compression coding on the abstract information tensor by using a memory network model to obtain a visual feature tensor, and then decode the visual feature tensor by using the memory network model to output feature coordinate information of a future movement trajectory of the person on an actual coordinate plane.
10. The system of claim 7, wherein the anomaly alarm module is configured to perform an early warning when the person is determined to be suspicious.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910921181.4A CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910921181.4A CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110674761A CN110674761A (en) | 2020-01-10 |
| CN110674761B true CN110674761B (en) | 2022-07-15 |
Family
ID=69079531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910921181.4A Active CN110674761B (en) | 2019-09-27 | 2019-09-27 | Regional behavior early warning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110674761B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449558A (en) * | 2020-03-26 | 2021-09-28 | 上海依图网络科技有限公司 | Method and device for monitoring abnormal behaviors of personnel |
| CN111935461A (en) * | 2020-09-11 | 2020-11-13 | 合肥创兆电子科技有限公司 | Based on intelligent security control system |
| CN113376656A (en) * | 2021-06-07 | 2021-09-10 | 重庆大学 | Multi-robot enclosure system based on LSTM prediction |
| CN115294515B (en) * | 2022-07-05 | 2023-06-13 | 南京邮电大学 | Comprehensive anti-theft management method and system based on artificial intelligence |
| CN115273370B (en) * | 2022-09-28 | 2023-01-03 | 苏州美集供应链管理股份有限公司 | System and method for monitoring personnel in real time based on visual scanning and track recognition |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509880A (en) * | 2018-03-21 | 2018-09-07 | 南京邮电大学 | A kind of video personage behavior method for recognizing semantics |
| CN109410496B (en) * | 2018-10-25 | 2022-04-01 | 北京交通大学 | Intrusion early warning method and device and electronic equipment |
-
2019
- 2019-09-27 CN CN201910921181.4A patent/CN110674761B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110674761A (en) | 2020-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110674761B (en) | Regional behavior early warning method and system | |
| JP6905850B2 (en) | Image processing system, imaging device, learning model creation method, information processing device | |
| CN106412501B (en) | A kind of the construction safety behavior intelligent monitor system and its monitoring method of video | |
| US9911294B2 (en) | Warning system and method using spatio-temporal situation data | |
| US9646228B2 (en) | Role-based tracking and surveillance | |
| Candamo et al. | Understanding transit scenes: A survey on human behavior-recognition algorithms | |
| JP4924607B2 (en) | Suspicious behavior detection apparatus and method, program, and recording medium | |
| CN109740411A (en) | Intelligent monitor system, monitoring method based on recognition of face and quickly go out alarm method | |
| Zin et al. | A Markov random walk model for loitering people detection | |
| CN110490126B (en) | Safe deposit box safety control system based on artificial intelligence | |
| Patil et al. | Suspicious movement detection and tracking based on color histogram | |
| CN116798176A (en) | Data management system based on big data and intelligent security | |
| Van Den Hengel et al. | Activity topology estimation for large networks of cameras | |
| Shivthare et al. | Suspicious activity detection network for video surveillance using machine learning | |
| CN112330742A (en) | Method and device for recording activity routes of key personnel in public area | |
| CN116778657A (en) | Method and system for intelligently identifying intrusion behavior | |
| Blauensteiner et al. | A socio-technical approach for event detection in security critical infrastructure | |
| Kapoor et al. | A study on various methods used in automatic video surveillance | |
| CN115294515B (en) | Comprehensive anti-theft management method and system based on artificial intelligence | |
| Prezioso et al. | Integrating Object Detection and Advanced Analytics for Smart City Crowd Management | |
| Gnanavel et al. | Smart Surveillance System and Prediction of Abnormal Activity in ATM Using Deep Learning | |
| Sharma et al. | Automated human detection and tracking for surveillance applications | |
| Pasha et al. | Artificial Intelligence Based Intruder Detection Home Security System | |
| Kumble et al. | Trajectory Movement Detection on ATM Survillance | |
| Salih et al. | Visual surveillance for hajj and umrah: a review |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |