CN116129490A - Monitoring device and monitoring method for complex environment behavior recognition - Google Patents
Monitoring device and monitoring method for complex environment behavior recognition Download PDFInfo
- Publication number
- CN116129490A CN116129490A CN202211596339.3A CN202211596339A CN116129490A CN 116129490 A CN116129490 A CN 116129490A CN 202211596339 A CN202211596339 A CN 202211596339A CN 116129490 A CN116129490 A CN 116129490A
- Authority
- CN
- China
- Prior art keywords
- module
- behavior
- face recognition
- monitoring
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a monitoring device and a monitoring method for complex environment behavior recognition, wherein the device comprises a super computer, a server, a monitoring camera, a face recognition camera, a PC (personal computer), a large display screen, a switch, a behavior recognition module, a face recognition module and an early warning module. The face recognition module is used for carrying out face recognition comparison according to the face photo data of the face recognition camera, verifying the identity information of personnel entering the operation place, and linking the early warning module to carry out recognition early warning on strangers; the early warning module is used for broadcasting abnormal behaviors according to the early warning information of the behavior recognition module and the face recognition module and sending display information; and the PC computer is used for acquiring the display information of the early warning module through the switch and controlling the display large screen to display abnormal behaviors of the display information. When unsafe behaviors occur, voice early warning is performed in real time in different areas and categories, and inspection efficiency is improved.
Description
Technical Field
The invention relates to the technical field of monitoring and recognition, in particular to a monitoring device and a monitoring method for complex environment behavior recognition.
Background
In recent years, along with the rapid development of social economy, the field safety management and control work only depends on means such as file requirements, field inspection and the like to improve the safety management level of the operation field, but the whole process cannot monitor the operation field for the characteristics of 'more points, wide range and indefinite period' of a complex construction field, and meanwhile, a large amount of manpower and material resources are also required to be consumed. The unsafe behavior identification of complicated construction sites is more and more emphasized, various dangerous operations such as overhead operation, hoisting operation, fire operation and the like are involved in the scaffold, cross operation possibly exists, risks are high, the space between the scaffolds is narrow, the structure is complex, light is dark, illumination is uneven, management difficulty is high, a technical means capable of intelligently identifying abnormal behaviors is urgently needed, a common monitoring device is that a computer is connected with a monitoring camera to identify behavior information, the computer is used for realizing behavior identification by simulating the motion profile of an object and comparing with a database, the identification target is relatively simple, but the identification rate is low, the false alarm rate is high under the complex environment, and the management requirement cannot be met. Meanwhile, when unsafe behaviors occur to the personnel on the construction site, the early warning information and the display are insufficient, and the effect of real-time warning cannot be achieved. Based on artificial intelligence technology in complex environment, the system realizes behavior recognition function, automatically researches and judges unsafe behaviors on site, and actively gives voice alarm, thereby realizing breakthrough of intelligent inspection and early warning.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a monitoring device and a monitoring method for complex environment behavior recognition, which can solve the technical problems that the existing method lacks a function of realizing behavior automatic recognition based on complex environment, automatically researches and judges unsafe behaviors on site, actively gives an alarm by voice and cannot realize intelligent inspection and early warning.
In order to achieve the above purpose, the present invention provides the following technical solutions:
in an embodiment of the application, a monitoring device for complex environment behavior recognition is provided, where the monitoring device includes a supercomputer, a server, a monitoring camera, a face recognition camera, a PC computer, a display large screen, a switch, a behavior recognition module, a face recognition module and an early warning module; the monitoring camera is connected to the behavior recognition module, the face recognition camera is connected to the face recognition module, and the super computer is connected to the behavior recognition module, the switch and the early warning module; the face recognition module, the server and the PC computer are connected to the switch, and the display large screen is connected to the PC computer;
the behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network and is used for judging the behavior of the video stream transmitted by the monitoring camera, analyzing the behavior by combining a deep learning model library and calculating a result;
the face recognition module is used for carrying out face recognition comparison according to the face photo data of the face recognition camera, verifying the identity information of personnel entering the operation place, and linking the early warning module to carry out recognition early warning on strangers;
the early warning module is used for broadcasting abnormal behaviors according to the early warning information of the behavior recognition module and the face recognition module and sending display information;
and the PC computer is used for acquiring the display information of the early warning module through the switch and controlling the display large screen to display abnormal behaviors of the display information.
In this embodiment, the behavior recognition module includes an abnormal behavior judgment module and an abnormal behavior verification module;
the abnormal behavior judging module is used for judging whether abnormal behaviors exist in a working place or not through a smoke extraction algorithm, a mobile phone playing algorithm, a safety helmet wearing-free algorithm and a safety belt wearing-free algorithm;
the abnormal behavior verification module is used for carrying out false alarm information verification on the abnormal behavior judgment information output by the abnormal behavior judgment module.
In this embodiment, the monitoring device further includes a storage module and a parameter configuration module;
the storage module is used for receiving the information transmitted by the switch and storing the output results of the behavior recognition module and the face recognition module;
the parameter configuration module is used for adjusting parameters of the face recognition camera and the monitoring camera, wherein the parameters comprise angles, focal lengths, confidence degrees and sensitivity.
In this embodiment, the face recognition camera and the monitoring camera each include an infrared sensor.
In this embodiment, the early warning module includes a plurality of intelligent speakers and a display information generating module;
the display information generation module is used for automatically editing text content according to the early warning information of the behavior recognition module and the face recognition module to generate broadcast information or display information for early warning abnormal behaviors;
the intelligent sound box is arranged in a plurality of operation areas and is used for broadcasting the broadcast information corresponding to the abnormal behaviors in each operation area in a text-to-speech mode.
In this embodiment, the comparison result of the face recognition module for face recognition comparison is displayed on the display large screen, and the display information on the display large screen includes a name, a gender, a snap photo and an affiliated team; and when the face recognition module cannot recognize the identity information of the personnel in the operation place, judging the person to be a stranger, linking the face recognition module to the early warning module to recognize and early warn the stranger, and sending early warning sound to the stranger through the display large screen.
In this embodiment, the convolutional neural network in the behavior recognition module includes an input layer, a convolutional layer, a pooling layer, a full link layer, and an output layer;
the training step of the convolutional neural network comprises the following steps: and manually calibrating each type of target violation behaviors and normal behavior image material samples in a complex environment through samples collected by video images, inputting the image material samples into a deep learning algorithm model for classifying and training unsafe behaviors, and outputting the algorithm model capable of identifying various unsafe behaviors as a convolutional neural network through continuous iterative training until convergence.
The application also provides a monitoring method for complex environment behavior recognition, which comprises the following steps:
setting the monitoring device for complex environment behavior recognition;
training a convolutional neural network capable of identifying various unsafe behaviors in a complex environment;
recording a monitoring video and synchronously identifying unsafe behavior;
and continuously performing behavior judgment and outputting a judgment result.
In this embodiment, the step of recording the surveillance video and identifying the unsafe behavior synchronously includes:
recording monitoring videos, wherein the monitoring cameras respectively transmit the monitoring videos in each operation area in the operation field to the super computer and store the monitoring videos;
the method comprises the steps of establishing a face recognition database, wherein the face recognition database stores the identity information of personnel in an operation place and photos of the personnel in the face recognition database, the personnel identity information comprises names, sexes and affiliated teams, the face recognition module carries out face recognition comparison according to face photo data of a face recognition camera, verifies the identity information of the personnel entering the operation place until all target personnel in a monitored area are monitored, and displays comparison results in linkage with the display large screen;
recording a motion trail, namely converting a monitoring video stream into a picture of a frame by the super computer through editing software, carrying out motion trail tracing according to a monitored object in the monitoring video, drawing a motion trail, establishing a behavior model and recording the motion trail;
a behavior recognition step, wherein the behavior recognition module recognizes unsafe behaviors of operators according to video streams generated in real time by the monitoring camera, and when the unsafe behaviors are monitored, the unsafe behaviors are linked with the early warning module, and classified and regional early warning is carried out;
and establishing a backup file, wherein the super computer stores the monitoring video and audio recorded by the monitoring camera and the captured photo in a compressed package mode.
In this embodiment, the step of continuously performing the behavior determination and outputting the determination result includes:
a step of confirming personnel information, in which the face recognition camera compares the obtained facial features with the data in the face recognition database one by one to confirm the personnel information, and outputs the face image with highest coincidence degree in the face recognition database as personnel information result of personnel comparison, and stores the captured personnel image into the database, and determines whether to permit entry according to the compared personnel information result, if the face recognition module can not recognize the personnel identity information of the operation place, the face recognition module judges the personnel to be stranger, and links with the early warning module to perform voice early warning;
and an unsafe behavior analysis step, wherein the super computer monitors whether unsafe behaviors exist in the working personnel in real time through a video stream of the monitoring camera, and when the unsafe behaviors are monitored, the super computer is linked with the early warning model, carries out real-time voice early warning in different areas and categories, displays pictures of the unsafe behaviors on a large display screen in real time, and switches to the monitoring camera for monitoring the unsafe behaviors, wherein the pictures of the monitoring camera are displayed on the large display screen.
The beneficial effects of the invention are as follows:
the invention relates to a behavior recognition technology under a complex environment based on a scaffold, wherein a behavior recognition algorithm is built in a super computer and is a synthetic algorithm, and a set of algorithms such as smoking, playing a mobile phone, wearing no safety helmet and wearing no safety belt is an algorithm model, so that abnormal behaviors are effectively recognized. The behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network and is used for judging the behavior of the video stream transmitted by the monitoring camera, and analyzing the behavior by combining a deep learning model library, so that the accuracy rate of behavior recognition can be greatly increased. When unsafe behaviors occur in a construction site, the site can realize regional and classified voice early warning in real time, and abnormal behaviors such as pictures of the unsafe behaviors are displayed on a large display screen in real time, so that the inspection efficiency of inspection personnel is improved, the unsafe behaviors of the construction site are really discovered, treated and solved at the first time, and the intelligent recognition of various unsafe behaviors in a complex environment is realized.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the following description will briefly explain the drawings used in the detailed description or the prior art descriptions, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort to those skilled in the art.
FIG. 1 is a schematic block diagram of a monitoring device for complex environmental behavior recognition in accordance with an embodiment of the present invention;
FIG. 2 is a flowchart of an identification method for training a deep learning algorithm model in a complex environment according to an embodiment of the present invention;
FIG. 3 is a block diagram of a monitoring device for complex environmental behavior recognition according to an embodiment of the present invention;
FIG. 4 is a flow chart of a monitoring method for complex environmental behavior recognition according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating steps for recording surveillance video and identifying unsafe behavior synchronously according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating steps for continuously performing behavior determination and outputting a determination result according to an embodiment of the present invention.
Detailed Description
The following describes the embodiments of the present invention further with reference to the drawings and examples. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
Example 1
The embodiment 1 of the invention provides a monitoring device and a monitoring method based on a behavior recognition technology capable of deep learning, namely a monitoring device and a monitoring method for complex environment behavior recognition. According to the application requirements of the scaffold operation area, a deep learning algorithm model is built, algorithm and calculation force resources are reasonably scheduled, and real-time acquisition and intelligent analysis of unstructured data such as videos and pictures are realized, and real-time early warning of large screens is realized through linkage broadcasting. The video monitoring traceability of the operation site in the complex operation environment of the scaffold is realized, real-time monitoring and early warning of illegal behaviors such as wearing no safety helmet, wearing no safety rope and illegal use, smoking, using a mobile phone and the like are realized, face recognition is carried out on personnel entering an area, and voice early warning is realized for strangers.
As shown in fig. 1, fig. 1 is a schematic block diagram of a monitoring device for complex environmental behavior recognition according to the present invention, where the monitoring device for complex environmental behavior recognition is based on a scaffold complex environmental behavior recognition technology, and includes a server, a PC computer, a supercomputer, a face recognition camera, a monitoring camera, a behavior recognition module, a face recognition module, an early warning module, a database, a supercomputer, a behavior recognition module, and a face recognition module, where the supercomputer receives information obtained by other modules, the behavior recognition module recognizes unsafe behavior, the face recognition module recognizes facial features, the database stores results of the behavior and the face recognition, the face recognition camera and the monitoring camera can record monitoring videos, the convolutional neural network can rapidly process the information, and once the unsafe behavior is confirmed, the supercomputer is linked with the early warning module, and displays the unsafe behavior early warning information on a large display screen, and the monitoring camera automatically transfers to the early warning monitor camera.
The super computer is connected with a server, a behavior recognition module, a face recognition module and an early warning module, and embedded with a behavior recognition algorithm and a convolutional neural network, the behavior recognition module is connected with an intelligent sound, a monitoring camera, a PC computer, a display large screen and the super computer, the face recognition module is embedded with a convolutional neural network and is connected with a PC computer, the face recognition module comprises an infrared lens and a light supplementing lamp, and the early warning module is connected with an intelligent sound and a monitoring camera and supports the linkage early warning signal of the IP address of the recognition camera.
The face recognition camera and the monitoring camera comprise infrared sensors, and the face recognition camera and the monitoring camera automatically sense and switch the daytime mode or the night mode. Aiming at the characteristics of complex scaffold environment, narrow space, dark light and uneven illumination, the face recognition camera and the monitoring camera comprise infrared sensors, automatically respond to the day and night, automatically filter light spots, and perform deep learning training model based on the personnel posture in the environment, so that false alarm is reduced.
The early warning module is respectively linked with the corresponding monitoring cameras, different voice information can be respectively broadcast to different abnormal behaviors in different areas, the described voice information can be broadcast in an mp3 format or in a text-to-voice format, and the text content can be edited by self.
The face recognition module of the face recognition camera is arranged in the camera, the face comparison result can be displayed on a large display screen, the display information can comprise contents such as names, sexes, snap shots, teams and the like, and when the face comparison result shows that strangers send out early warning sounds through the large display screen.
And parameters such as angle, focal length, confidence coefficient, sensitivity and the like of the camera are adjusted in a narrow space of the scaffold, so that the recognition efficiency is improved, false alarm is reduced, and the optimal recognition effect is achieved.
The monitoring method for complex environment behavior recognition is used for recognizing unsafe behaviors of operators, including not wearing safety helmets, smoking, playing mobile phones, not tying safety belts and the like, and once the unsafe behaviors of the operators occur, the early warning module is started immediately, corresponding unsafe behavior voices are broadcast by intelligent sound in corresponding areas, and voice setting can be broadcast according to voice or text-to-voice functions. The unsafe behavior algorithm is built in the super computer, the video stream of the monitoring camera is analyzed in real time, and an early warning event occurs once the confidence level of unsafe behavior of workers reaches a set threshold value.
Specifically, the super computer implementation process comprises the following specific steps:
the first step: initializing monitoring equipment, a plurality of surveillance cameras, a face identification camera, a supercomputer, switch, PC, show large-scale screen, a plurality of intelligent stereo sets have formed a whole set monitored control system, and intelligent stereo sets divide according to multizone, and face identification camera sets up in the entrance, sets up personnel identification camera number according to the entry volume.
And a second step of: training of deep learning algorithm models in complex environments, as shown in fig. 2, identifies a method flow chart. The method comprises the steps that a deep learning algorithm model marks various unsafe behaviors based on field pictures of video streams, a behavior algorithm model is built, and the algorithm model is embedded into a super computer; the recognition effect is optimal by adjusting the angle, focal length, recognition sensitivity, placement degree and the like of the camera, the model can be updated and iterated through the addition of the data set, the model is more accurate, and the recognition model can be more suitable for the current scene through continuous learning optimization under the scaffold scene.
And a third step of: recording a monitoring video and identifying unsafe behavior.
(a) Recording a monitoring video, wherein the monitoring cameras are arranged at positions of different layers and different channels of the scaffold to record the monitoring video, the monitoring cameras respectively transmit the monitoring video into the super computer, and the super computer stores the video recorded by the monitoring cameras into a storage hard disk of the super computer;
(b) The method comprises the steps of establishing a face recognition database, adding a photo of a person into a model database by the face recognition module in a face recognition camera, adding information of the person including name, gender, departments and the like into the model database, repeating the step of face comparison until all target numbers in a monitoring area are monitored completely, and linking with a display large screen;
(c) Recording a motion trail, converting a monitoring video stream into a frame of picture by the super computer through editing software, carrying out motion trail tracing according to a monitoring object in the monitoring video, drawing the motion trail, and establishing a behavior model;
(d) The behavior recognition module recognizes unsafe behaviors of operators according to video streams generated in real time by the monitoring camera, and links with the early warning module once the unsafe behaviors are recognized and monitored, and early warning is carried out by intelligent sound classification and zoning;
(e) Establishing a backup file: the super computer stores the monitoring video, the audio and the captured pictures recorded by the monitoring camera into a storage hard disk in a compressed package mode;
fourth step: and judging and outputting the behaviors.
(a) Personnel information confirmation: and the face recognition camera compares the facial features with the data in the face recognition database one by one to confirm personnel information, the most conforming face image in the face recognition database is used as personnel information result of personnel comparison, the snap shot personnel images are stored in the database, whether the access is permitted or not is determined according to the compared personnel information result, and if the comparison is unsuccessful, the voice early warning is carried out in linkage with the early warning module.
(b) Unsafe behavior analysis: the super computer is embedded with a behavior algorithm model, the algorithm model establishes an unsafe behavior data model, whether unsafe behaviors exist in operators or not is monitored in real time through video streams of the monitoring camera, once the unsafe behaviors are monitored, the early warning model is immediately linked, real-time voice early warning is performed in different areas and categories, pictures of the unsafe behaviors are displayed on a large display screen in real time, and the pictures are immediately switched to the monitoring camera for monitoring the unsafe behaviors, and the pictures of the monitoring camera are displayed on the large display screen.
As shown in fig. 2, fig. 2 is a flowchart of an identification method of deep learning algorithm model training in a complex environment. The method comprises the steps that an unsafe behavior of a deep learning algorithm model is marked based on a field picture of a video stream, a behavior algorithm model is built, and the algorithm model is embedded into a super computer; the model can be updated and iterated through the addition of the data set, becomes more accurate, and is more suitable for the current scene through continuous optimization.
And constructing a convolutional neural network based on unsafe behavior, designing and setting basic parameters of a model, and inputting the generated data set into the model for training and evaluation. Firstly, manually calibrating image material samples of each type of target illegal behaviors and normal behaviors in a complex environment of a scaffold through samples collected by video images, manually calibrating each behavior in the material samples, classifying and training the current unsafe behaviors by using a model algorithm, outputting an algorithm model capable of identifying various unsafe behaviors, and continuously iterating through the algorithm based on the complex environment of the scaffold to adapt to the complex environment of the scaffold.
The behavior recognition module is embedded in the super computer, performs behavior recognition based on the video stream according to the deep learning algorithm model, and can perform setting according to scenes by enabling the fastest continuous practice setting of unsafe behaviors to be recognized for 1s or enabling the unsafe behaviors to last for 1s to 10 s.
The beneficial effects of the invention are as follows:
the invention relates to a behavior recognition technology under a complex environment based on a scaffold, wherein a behavior recognition algorithm is built in a super computer and is a synthetic algorithm, namely, smoking, playing a mobile phone, not wearing a safety helmet and not fastening a safety belt are integrated into an algorithm model, so that GPU (graphic processing unit) resources of the super computer are saved. The training of the deep learning model is carried out according to the field environment of the scaffold, so that the accuracy of behavior recognition can be greatly increased.
When unsafe behaviors occur in a construction site, the site can be subjected to real-time regional and classified voice early warning, pictures of the unsafe behaviors are displayed on a large display screen in real time, and the pictures are immediately switched to a monitoring camera for monitoring the unsafe behaviors, and the pictures of the monitoring camera are displayed on the large display screen without manually switching the pictures of the monitoring camera. The inspection efficiency of inspection personnel is improved, the first time discovery, treatment and solution of unsafe behaviors of a construction site are truly achieved, and the intelligent identification of various unsafe behaviors in a complex environment is realized by using a deep learning method.
Example 2
Based on the same inventive concept as that of embodiment 1, all technical features of embodiment 1 are included in embodiment 2.
As shown in fig. 1 and 3, in the present embodiment, a monitoring apparatus 100 for complex environmental behavior recognition is provided, the monitoring apparatus 100 includes a supercomputer 1, a server 2, a monitoring camera 3, a face recognition camera 4, a PC computer 5, a display screen 6, a switch 7, a behavior recognition module 8, a face recognition module 9, and an early warning module 10; the monitoring camera 3 is connected to the behavior recognition module 8, the face recognition camera 4 is connected to the face recognition module 9, and the super computer 1 is connected to the behavior recognition module 8, the switch 7 and the early warning module 10; the face recognition module 9, the server 2 and the PC computer 5 are connected to the switch 7, and the display screen 6 is connected to the PC computer 5.
The behavior recognition module 8 is embedded with a behavior recognition algorithm and a convolutional neural network and is used for judging the behavior of the video stream transmitted by the monitoring camera 3, analyzing the behavior by combining a deep learning model library and calculating a result.
The face recognition module 9 includes a face recognition comparison module 91, which is configured to perform face recognition comparison according to the face photo data of the face recognition camera 4, verify identity information of personnel entering the workplace, and link the early warning module 10 to perform recognition early warning on strangers.
The early warning module 10 is configured to broadcast abnormal behaviors according to early warning information of the behavior recognition module 8 and the face recognition module 9, and send display information.
The PC computer 5 is used for acquiring the display information of the early warning module 10 through the switch 7 and controlling the display large screen 6 to display abnormal behaviors of the display information.
As shown in fig. 3, in the present embodiment, the behavior recognition module 8 includes an abnormal behavior judgment module 81 and an abnormal behavior verification module 82.
The abnormal behavior judging module 81 is used for judging whether abnormal behaviors exist in the operation field or not through a smoke exhausting algorithm, a mobile phone playing algorithm, a safety helmet wearing algorithm and a safety belt wearing algorithm.
The abnormal behavior verification module 82 is configured to perform false alarm information verification on the abnormal behavior judgment information output by the abnormal behavior judgment module 81.
As shown in fig. 3, in the present embodiment, the monitoring apparatus 100 further includes a storage module 11 and a parameter configuration module 12; the storage module 11 is preferably a database provided in the supercomputer 1 or a database provided in the server 2. The parameter configuration module 12 may be provided in the supercomputer 1 or in the server 2, or may be provided separately.
The storage module 11 is configured to receive information transmitted by the switch 7, and store output results of the behavior recognition module 8 and the face recognition module 9.
The parameter configuration module 12 is configured to adjust parameters of the face recognition camera 4 and the monitoring camera 3, where the parameters include an angle, a focal length, a confidence level, and a sensitivity.
In the present embodiment, the face recognition camera 4 and the monitoring camera 3 each include an infrared sensor. In order to adapt to the condition of poor light of the complex environment, the face recognition camera 4 comprises an infrared lens and a light supplementing lamp.
As shown in fig. 3, in this embodiment, the pre-warning module 10 includes a plurality of smart speakers 101 and a display information generating module 102.
The display information generating module 102 is configured to automatically edit text content according to the early warning information of the behavior recognizing module 8 and the face recognition module 9 to generate broadcast information or display information for early warning of abnormal behaviors.
The intelligent sound box 101 is arranged in a plurality of operation areas, and the intelligent sound box 101 is used for broadcasting the broadcast information corresponding to the abnormal behaviors in each operation area in a text-to-speech mode.
In this embodiment, the comparison result of the face recognition module 9 for face recognition comparison is displayed on the display large screen 6, and the display information on the display large screen 6 includes name, gender, snap photo and belonging team; and when the face recognition module 9 cannot recognize the identity information of the personnel in the operation place, the person recognition module 9 determines that the person is a stranger, the early warning module 10 is linked to recognize and early warn the stranger, and the early warning sound is sent out to the stranger through the large display screen 6.
As shown in fig. 2, in the present embodiment, the convolutional neural network in the behavior recognition module 8 includes an input layer, a convolutional layer, a pooling layer, a full link layer, and an output layer;
as shown in fig. 2, the training step of the convolutional neural network includes: and manually calibrating each type of target violation behaviors and normal behavior image material samples in a complex environment through samples collected by video images, inputting the image material samples into a deep learning algorithm model for classifying and training unsafe behaviors, and outputting the algorithm model capable of identifying various unsafe behaviors as a convolutional neural network through continuous iterative training until convergence.
As shown in fig. 4, the present application further provides a monitoring method for complex environmental behavior recognition, including the following steps:
s1, setting the monitoring device 100 for complex environment behavior recognition;
s2, training a convolutional neural network capable of identifying various unsafe behaviors in a complex environment;
s3, recording a monitoring video and synchronously identifying unsafe behaviors;
s4, continuously performing behavior judgment and outputting a judgment result.
As shown in fig. 5, in this embodiment, the step of recording the surveillance video and identifying unsafe behavior synchronously includes:
s31, recording monitoring videos, wherein the monitoring camera 3 respectively transmits the monitoring videos in each operation area in the operation field to the super computer 1 and stores the monitoring videos;
s32, a face recognition database is established, the identity information of the personnel in the operation place and the photos of the personnel are stored in the face recognition database, the personnel identity information comprises names, sexes and groups to which the personnel belong, the face recognition module 9 performs face recognition comparison according to the face photo data of the face recognition camera 4, verifies the identity information of the personnel entering the operation place until all target personnel in the monitored area are monitored, and displays comparison results in a linkage way with the display screen 6;
s33, recording a motion trail, namely converting a monitoring video stream into a picture of a frame by clipping software by the super computer 1, carrying out motion trail tracing according to a monitoring object in the monitoring video, drawing a motion trail, and establishing a behavior model to record the motion trail;
s34, a behavior recognition step, wherein the behavior recognition module 8 recognizes unsafe behaviors of operators according to video streams generated in real time by the monitoring camera, and when the unsafe behaviors are monitored, the unsafe behaviors are linked with the early warning module 10 to perform classified and regional early warning;
and S35, establishing a backup file, wherein the super computer 1 stores the monitoring video and audio recorded by the monitoring camera 3 and the snap shot photos in a compressed package mode.
As shown in fig. 6, in this embodiment, the steps of continuously performing behavior determination and outputting a determination result include:
s41, personnel information confirmation, namely the face recognition camera 4 compares the acquired facial features with the data in the face recognition database one by one to confirm personnel information, outputs the face image with highest coincidence degree in the face recognition database as personnel information result of personnel comparison, stores the snap shot personnel images into the database, determines whether to permit entering according to the compared personnel information result, judges that the person is a stranger when the face recognition module 9 cannot recognize the person identity information as the operation place, and is linked with the early warning module 10 to perform voice early warning;
s42, an unsafe behavior analysis step, wherein the super computer 1 monitors whether unsafe behaviors exist in the working personnel in real time through a video stream of the monitoring camera, and when the unsafe behaviors are monitored, the pre-warning model is linked, real-time voice pre-warning is carried out in different areas and categories, pictures of the unsafe behaviors are displayed on the display large screen 6 in real time, and the pictures are switched to the monitoring camera 3 for monitoring the unsafe behaviors, wherein the pictures of the monitoring camera 3 are displayed on the display large screen 6.
The beneficial effects of the invention are as follows:
the invention relates to a behavior recognition technology under a complex environment based on a scaffold, wherein a behavior recognition algorithm is built in a super computer and is a synthetic algorithm, and a set of algorithms such as smoking, playing a mobile phone, wearing no safety helmet and wearing no safety belt is an algorithm model, so that abnormal behaviors are effectively recognized. The behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network and is used for judging the behavior of the video stream transmitted by the monitoring camera, and analyzing the behavior by combining a deep learning model library, so that the accuracy rate of behavior recognition can be greatly increased. When unsafe behaviors occur in a construction site, the site can realize regional and classified voice early warning in real time, and abnormal behaviors such as pictures of the unsafe behaviors are displayed on a large display screen in real time, so that the inspection efficiency of inspection personnel is improved, the unsafe behaviors of the construction site are really discovered, treated and solved at the first time, and the intelligent recognition of various unsafe behaviors in a complex environment is realized.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that it will be apparent to those skilled in the art that several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the scope of the invention.
Claims (10)
1. A monitoring device for complex environmental behavior recognition, characterized in that:
the monitoring device comprises a super computer, a server, a monitoring camera, a face recognition camera, a PC computer, a large display screen, a switch, a behavior recognition module, a face recognition module and an early warning module; the monitoring camera is connected to the behavior recognition module, the face recognition camera is connected to the face recognition module, and the super computer is connected to the behavior recognition module, the switch and the early warning module; the face recognition module, the server and the PC computer are connected to the switch, and the display large screen is connected to the PC computer;
the behavior recognition module is embedded with a behavior recognition algorithm and a convolutional neural network and is used for judging the behavior of the video stream transmitted by the monitoring camera, analyzing the behavior by combining a deep learning model library and calculating a result;
the face recognition module is used for carrying out face recognition comparison according to the face photo data of the face recognition camera, verifying the identity information of personnel entering the operation place, and linking the early warning module to carry out recognition early warning on strangers;
the early warning module is used for broadcasting abnormal behaviors according to the early warning information of the behavior recognition module and the face recognition module and sending display information;
and the PC computer is used for acquiring the display information of the early warning module through the switch and controlling the display large screen to display abnormal behaviors of the display information.
2. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the behavior recognition module comprises an abnormal behavior judgment module and an abnormal behavior verification module;
the abnormal behavior judging module is used for judging whether abnormal behaviors exist in a working place or not through a smoke extraction algorithm, a mobile phone playing algorithm, a safety helmet wearing-free algorithm and a safety belt wearing-free algorithm;
the abnormal behavior verification module is used for carrying out false alarm information verification on the abnormal behavior judgment information output by the abnormal behavior judgment module.
3. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the monitoring device also comprises a storage module and a parameter configuration module;
the storage module is used for receiving the information transmitted by the switch and storing the output results of the behavior recognition module and the face recognition module;
the parameter configuration module is used for adjusting parameters of the face recognition camera and the monitoring camera, wherein the parameters comprise angles, focal lengths, confidence degrees and sensitivity.
4. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the face recognition camera and the monitoring camera both comprise infrared sensors.
5. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the early warning module comprises a plurality of intelligent sound boxes and a display information generation module;
the display information generation module is used for automatically editing text content according to the early warning information of the behavior recognition module and the face recognition module to generate broadcast information or display information for early warning abnormal behaviors;
the intelligent sound box is arranged in a plurality of operation areas and is used for broadcasting the broadcast information corresponding to the abnormal behaviors in each operation area in a text-to-speech mode.
6. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the comparison result of the face recognition module for face recognition comparison is displayed on the display large screen, and the display information on the display large screen comprises names, sexes, snap shots and groups of groups; and when the face recognition module cannot recognize the identity information of the personnel in the operation place, judging the person to be a stranger, linking the face recognition module to the early warning module to recognize and early warn the stranger, and sending early warning sound to the stranger through the display large screen.
7. A monitoring device for complex environmental behavior recognition as defined in claim 1, wherein: the convolutional neural network in the behavior recognition module comprises an input layer, a convolutional layer, a pooling layer, a full-link layer and an output layer;
the training step of the convolutional neural network comprises the following steps: and manually calibrating each type of target violation behaviors and normal behavior image material samples in a complex environment through samples collected by video images, inputting the image material samples into a deep learning algorithm model for classifying and training unsafe behaviors, and outputting the algorithm model capable of identifying various unsafe behaviors as a convolutional neural network through continuous iterative training until convergence.
8. A monitoring method for complex environmental behavior recognition, comprising:
providing a monitoring device for complex environmental behavior recognition according to any one of claims 1 to 7; training a convolutional neural network capable of identifying various unsafe behaviors in a complex environment;
recording a monitoring video and synchronously identifying unsafe behavior;
and continuously performing behavior judgment and outputting a judgment result.
9. The method for monitoring complex environmental behavior recognition according to claim 8, wherein the step of recording the monitoring video and synchronously recognizing unsafe behavior comprises:
recording monitoring videos, wherein the monitoring cameras respectively transmit the monitoring videos in each operation area in the operation field to the super computer and store the monitoring videos;
the method comprises the steps of establishing a face recognition database, wherein the face recognition database stores the identity information of personnel in an operation place and photos of the personnel in the face recognition database, the personnel identity information comprises names, sexes and affiliated teams, the face recognition module carries out face recognition comparison according to face photo data of a face recognition camera, verifies the identity information of the personnel entering the operation place until all target personnel in a monitored area are monitored, and displays comparison results in linkage with the display large screen;
recording a motion trail, namely converting a monitoring video stream into a picture of a frame by the super computer through editing software, carrying out motion trail tracing according to a monitored object in the monitoring video, drawing a motion trail, establishing a behavior model and recording the motion trail;
a behavior recognition step, wherein the behavior recognition module recognizes unsafe behaviors of operators according to video streams generated in real time by the monitoring camera, and when the unsafe behaviors are monitored, the unsafe behaviors are linked with the early warning module, and classified and regional early warning is carried out;
and establishing a backup file, wherein the super computer stores the monitoring video and audio recorded by the monitoring camera and the captured photo in a compressed package mode.
10. The method for monitoring behavior recognition in a complex environment according to claim 8, wherein the step of continuously performing behavior judgment and outputting the judgment result comprises:
a step of confirming personnel information, in which the face recognition camera compares the obtained facial features with the data in the face recognition database one by one to confirm the personnel information, and outputs the face image with highest coincidence degree in the face recognition database as personnel information result of personnel comparison, and stores the captured personnel image into the database, and determines whether to permit entry according to the compared personnel information result, if the face recognition module can not recognize the personnel identity information of the operation place, the face recognition module judges the personnel to be stranger, and links with the early warning module to perform voice early warning;
and an unsafe behavior analysis step, wherein the super computer monitors whether unsafe behaviors exist in the working personnel in real time through a video stream of the monitoring camera, and when the unsafe behaviors are monitored, the super computer is linked with the early warning model, carries out real-time voice early warning in different areas and categories, displays pictures of the unsafe behaviors on a large display screen in real time, and switches to the monitoring camera for monitoring the unsafe behaviors, wherein the pictures of the monitoring camera are displayed on the large display screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211596339.3A CN116129490A (en) | 2022-12-13 | 2022-12-13 | Monitoring device and monitoring method for complex environment behavior recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211596339.3A CN116129490A (en) | 2022-12-13 | 2022-12-13 | Monitoring device and monitoring method for complex environment behavior recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116129490A true CN116129490A (en) | 2023-05-16 |
Family
ID=86299956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211596339.3A Pending CN116129490A (en) | 2022-12-13 | 2022-12-13 | Monitoring device and monitoring method for complex environment behavior recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116129490A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118072375A (en) * | 2024-04-17 | 2024-05-24 | 云燕科技(山东)股份有限公司 | Face image acquisition system |
-
2022
- 2022-12-13 CN CN202211596339.3A patent/CN116129490A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118072375A (en) * | 2024-04-17 | 2024-05-24 | 云燕科技(山东)股份有限公司 | Face image acquisition system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109300471B (en) | Intelligent video monitoring method, device and system for field area integrating sound collection and identification | |
| JP7036186B2 (en) | Video processing system, video processing method and video processing program | |
| TWI749113B (en) | Methods, systems and computer program products for generating alerts in a video surveillance system | |
| CN211293956U (en) | AI-based identification and alarm system for abnormal agent on construction site | |
| CN110223208A (en) | A kind of garden safety monitoring system and method | |
| CN110516529A (en) | It is a kind of that detection method and system are fed based on deep learning image procossing | |
| KR102407327B1 (en) | Apparatus for Monitoring Fire And System having the same | |
| KR20200052418A (en) | Automated Violence Detecting System based on Deep Learning | |
| KR102511287B1 (en) | Image-based pose estimation and action detection method and appratus | |
| CN116165981A (en) | Intelligent monitoring system for industrial industry safety production | |
| Fawzi et al. | Embedded real-time video surveillance system based on multi-sensor and visual tracking | |
| CN114187541A (en) | Intelligent video analysis method and storage device for user-defined service scene | |
| CN115019462A (en) | Video processing method, device, storage medium and equipment | |
| CN110928305B (en) | Patrol method and system for patrol robot of railway passenger station | |
| CN110602453B (en) | Internet of things big data intelligent video security monitoring system | |
| CN115567690A (en) | Intelligent monitoring system capable of automatically identifying dangerous points of field operation | |
| CN111860187A (en) | High-precision worn mask identification method and system | |
| CN113392706A (en) | Device and method for detecting smoking and using mobile phone behaviors | |
| CN210091231U (en) | Wisdom garden management system | |
| CN116129490A (en) | Monitoring device and monitoring method for complex environment behavior recognition | |
| CN116152745A (en) | Smoking behavior detection method, device, equipment and storage medium | |
| KR20130047131A (en) | Method and system for surveilling contents of surveillance using mobile terminal | |
| WO2024124970A1 (en) | Monitoring apparatus and method for performing behavior recognition in complex environment | |
| KR100902275B1 (en) | Cctv system for intelligent security and method thereof | |
| CN114511817A (en) | Micro-space-oriented intelligent supervision system for panoramic portrait of personnel behaviors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |