CN113850229A - Method and system for early warning abnormal behaviors of people based on video data machine learning and computer equipment - Google Patents
Method and system for early warning abnormal behaviors of people based on video data machine learning and computer equipment Download PDFInfo
- Publication number
- CN113850229A CN113850229A CN202111211548.7A CN202111211548A CN113850229A CN 113850229 A CN113850229 A CN 113850229A CN 202111211548 A CN202111211548 A CN 202111211548A CN 113850229 A CN113850229 A CN 113850229A
- Authority
- CN
- China
- Prior art keywords
- data
- video
- early warning
- abnormal
- person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 206010000117 Abnormal behaviour Diseases 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000010801 machine learning Methods 0.000 title claims description 32
- 230000006399 behavior Effects 0.000 claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 19
- 230000005021 gait Effects 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims abstract description 13
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 230000033001 locomotion Effects 0.000 claims abstract description 11
- 238000004140 cleaning Methods 0.000 claims abstract description 6
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000012544 monitoring process Methods 0.000 claims description 33
- 230000002159 abnormal effect Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 16
- 230000005540 biological transmission Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 238000012806 monitoring device Methods 0.000 claims description 7
- 238000000926 separation method Methods 0.000 claims description 4
- 230000005856 abnormality Effects 0.000 claims 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000007726 management method Methods 0.000 description 6
- 230000006996 mental state Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 230000008921 facial expression Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000002787 reinforcement Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000008451 emotion Effects 0.000 description 2
- 230000002996 emotional effect Effects 0.000 description 2
- 238000013210 evaluation model Methods 0.000 description 2
- 238000013401 experimental design Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000033772 system development Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010063659 Aversion Diseases 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000000586 desensitisation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000001766 physiological effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Abstract
The invention relates to the field of artificial intelligence, in particular to a method, a system and computer equipment for early warning abnormal behaviors of personnel, wherein the method comprises the steps of acquiring video image data and cleaning the video image data; preprocessing video image data; identifying human motion modes, setting a classification label for each type, and performing multi-mode feature coding; removing redundant features of the data center and selecting the features through information gain; constructing a tensor neural network to track people appearing in the video, carrying out gait detection based on a gait energy map according to a tracked task outline, carrying out skin exposure detection on the task in the video, taking the exposure degree as an appearance clothing index of the people, calculating video frame images for correlation by integrating the indexes, and carrying out behavior abnormity judgment on the people according to the correlation calculation result; the invention can improve the working efficiency of field management and control, reduce the working load and enhance the early warning error control.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a method, a system and computer equipment for early warning abnormal behaviors of people based on video data machine learning.
Background
Abnormal behaviors occur in people in a smart park (a hospital, a campus, a construction site and the like), and some people even do not need to pay the cost of life (such as self abandonment of a building and fierce murder of others). Data such as video image monitoring are fully utilized in the information-based construction of the intelligent park, abnormal behaviors of personnel in the park are analyzed, and decision support in the aspects of early warning and the like is provided in time. Based on relevant artificial intelligence technologies such as ontology knowledge, reinforcement learning and three-dimensional human posture estimation, a new technology is developed to improve the decision support capability of monitoring video images of park personnel, and particularly improve the prevention and control capability of abnormal behavior risks (particularly life self-abandoning behaviors and the like).
Negative and abnormal upsets are the abnormal behavior states of major concern for this system. After face detection and video image data reinforcement learning, feature extraction such as appearance clothes, body state, gait, pace, facial expression, mental state and the like is carried out, and three types (normal, deep and abnormally high) of emotional states of people are mainly judged. Since most of the life-victims will express their intentions in the form of emotions, behaviors, etc., these are the help-seeking signals that are not ignored. The passive state is mainly represented by the negative characteristic information of sadness, depression, lonely, helplessness, despair and the like which are exposed for a long time. The abnormally high level is manifested as a sudden, unprovoked, full motivation. For example, from the trajectory extracted from the video image data, it is analyzed that a person looks calm at ordinary times, suddenly shows a strong passion and a heavy feeling on a certain day, but it is understood that nothing of particular importance has recently happened to the person by making close contact with the person, or the like. Then the person belongs to an abnormal behavior state due to the provision of alert information in the early warning decision support system.
Disclosure of Invention
In order to predict the abnormal behaviors of people in a park in a specific park and reduce the occurrence of special events, the invention provides a method, a system and computer equipment for early warning the abnormal behaviors of the people based on video data machine learning, wherein the method comprises the following steps:
s1, acquiring video image data and cleaning the video image data;
s2, preprocessing the video image data, including removing information related to privacy in the data and adopting a frame quality enhancement method for the fuzzy video to improve the video definition;
s3, identifying the human motion mode, setting a classification label for each type, and performing multi-mode feature coding;
s4, performing feature engineering processing on the multi-mode codes, namely removing redundant features of the data center and selecting the features through information gain;
s5, constructing a tensor neural network to track people appearing in the video, carrying out gait detection based on a gait energy map according to tracked task contours, carrying out skin exposure detection on tasks in the video, taking the exposure degree as an appearance clothing index of the people, calculating video interframe images for correlation by integrating the indexes, and carrying out behavior abnormity judgment on the people according to the correlation calculation result; s6, early warning decision support is constructed, namely if the dynamic information of similarity and difference between the inter-frame images is acquired to show that the behavior of the shot person is abnormal, the shot person is informed to deal with the dynamic information.
Further, after the characteristic engineering processing is performed in step S3, the data is detected, and if there is non-random missing data and data is made, the data is inferred by using a statistical machine learning method and dual robustness to obtain complete data of the data.
Further, the multi-modal feature coding specifically includes the following processes: using a 5-dimensional tensor structure for the acquired image, and encoding video data through (samples, frames, height, width, channels), wherein the 5-dimensional tensor data respectively represent samples, frames, height, width and channels of the video data; the acquired video image is subjected to a 4-dimensional tensor structure, and video data are coded through (samples, heights, widths, channels), so that 4-dimensional tensor data of samples, heights, widths and channels of the video data are respectively represented.
Further, tracking people present in the video includes:
taking the multi-modal coded image as the input of a tensor neural network;
the tensor neural network comprises human posture recognition, pedestrian detection and foreground and background separation;
acquiring a posture key sequence point of a human body in an image through human body posture identification;
detecting a person through pedestrian detection, segmenting a task from an image, and acquiring a posture outline sequence of the person by combining the separation of a foreground and a background;
and tracking and identifying the person through the acquired pose key sequence points and the pose outline sequence.
Further, the skin exposure degree of the task in the video is detected, the exposure degree is used as the appearance clothing index process of the person, namely the skin area of the person in the image is obtained based on the elliptical skin model, and the proportion of the exposed skin area to the outline of the person is used as the appearance clothing index.
Furthermore, after gait detection and appearance clothing indexes are carried out, a sliding window with the length of N frames and the step length of t is set, the gait detection and appearance clothing indexes are valued through the sliding window, and the average value of data of each window is taken as data for correlation calculation.
Further, when the difference judgment is carried out, the correlation between the two images is calculated through the Euclidean distance.
Furthermore, when the behavior abnormity of the human is judged, if the similarity of the two images extracted in the S frame is lower than a certain threshold value, the task abnormity is judged.
The invention also provides a personnel abnormal behavior early warning system which comprises a front terminal system, a transmission network and a monitoring center, wherein the front terminal system comprises various monitoring equipment, the front terminal system transmits information acquired by the monitoring equipment to the monitoring center through the transmission network, the monitoring center comprises an abnormal behavior prediction device, the device predicts whether the person to be shot is abnormal or not through the collected information, if so, the monitoring center gives an alarm and projects the monitoring equipment capable of shooting the person onto a monitor of the monitoring center for real-time monitoring.
The invention also provides a computer device for early warning the abnormal behavior of the personnel, which comprises a memory and a processor, wherein the memory is used for storing any one of the early warning methods for the abnormal behavior of the personnel, and the processor operates the method stored in the memory to predict whether the behavior of the personnel in the monitor is abnormal.
The invention also provides a personnel abnormal behavior early warning system which comprises a front terminal system, a transmission network and a monitoring center, wherein the front terminal system comprises various monitoring equipment, the front terminal system transmits information collected by the monitoring equipment to the monitoring center through the transmission network, the monitoring center comprises an abnormal behavior prediction device, the device predicts whether the person to be shot is abnormal or not through the collected information, if yes, the monitoring center gives an alarm and projects the monitoring equipment capable of shooting the person onto a monitor of the monitoring center for real-time monitoring.
The invention also provides a computer device for early warning the abnormal behavior of the personnel, which comprises a memory and a processor, wherein the memory is used for storing any one of the above-mentioned methods for early warning the abnormal behavior of the personnel, and the processor operates the method stored in the memory to predict whether the behavior of the personnel in the monitor is abnormal.
The invention constructs an abnormal behavior pattern classification model set based on machine learning, and carries out the method of identifying the abnormal behavior according to different granularities of human behavior characteristics, thereby covering the main requirements of matching models of human face data, identifying walking gait and evaluating mental state, improving the intelligent level of an early warning decision support system, and highlighting the important role in monitoring video image data utilization and multiplexing; through the processes of video image sampling, preprocessing, coding, feature extraction, data association and the like, an early warning database based on video images is constructed, technical support is provided for knowledge reasoning and data management of high-dimensional abnormal behavior identification data, the field management and control working efficiency can be improved, the workload can be reduced, early warning error control can be enhanced, the service mode of an intelligent park information sharing platform is enriched, and the application value of an early warning control panel is improved.
Drawings
FIG. 1 is a schematic diagram of a live video image monitoring network architecture according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an abnormal behavior early warning process based on machine learning of surveillance video images according to an embodiment of the present invention;
fig. 3 is a schematic diagram of another abnormal behavior early warning process based on machine learning of surveillance video images according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of information interaction of abnormal behavior early warning based on machine learning of surveillance video images, according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an abnormal behavior early warning apparatus for machine learning of surveillance video images according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another abnormal behavior early warning device for machine learning of surveillance video images according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a video data machine learning-based personnel abnormal behavior early warning method, which specifically comprises the following steps:
s1, acquiring video image data and cleaning the video image data;
s2, preprocessing the video image data, including removing information related to privacy in the data and adopting a frame quality enhancement method for the fuzzy video to improve the video definition;
s3, identifying the human motion mode, setting a classification label for each type, and performing multi-mode feature coding;
s4, performing feature engineering processing on the multi-mode codes, namely removing redundant features of the data center and selecting the features through information gain;
s5, a mental state evaluation model, a walking gait recognition model and a face data matching model, and constructing a tensor neural network to perform correlation calculation on key semantic regions between frames;
and S6, constructing early warning decision support, namely informing relevant personnel to deal with the abnormal behavior of the shot object.
Example 1
The embodiment provides a personnel abnormal behavior early warning system based on video data machine learning, including front end subsystem, transmission network and surveillance center, wherein front end subsystem includes various supervisory equipment, front end subsystem conveys the information that supervisory equipment gathered to the surveillance center through transmission network, the surveillance center includes unusual behavior prediction device, the device is taken a photograph whether the personnel exist unusually through the information prediction of collecting, if exist then the surveillance center sends out the police dispatch newspaper and will shoot on the supervisory equipment display of surveillance center of this personnel, carry out real-time supervision.
The front terminal system in the embodiment is mainly responsible for field image acquisition, video storage, alarm receiving and sending, data acquisition of other sensors and network transmission; the transmission network mainly realizes data uploading between the operation site and the monitoring center through a special line and the Internet; the monitoring center is a place for executing daily monitoring, system management and emergency early warning command. The functions of remote video monitoring of an operation site, remote rotation of a cloud control ball machine, remote receiving of site alarm, remote voice conversation command with the site and the like are realized through a site monitoring and controlling system; the manager can know the early warning and emergency management and control process on site in real time. The real-time monitoring camera is managed, and meanwhile, the functions of backup, query and playback of monitoring images according to information such as camera numbers, time, events and the like are supported through the hard disk video recorder. And multiple pictures are simultaneously played back on the same display, and up to 25 pictures are simultaneously played back synchronously. And the remote real-time control of the camera holder and the lens is supported. The system and the device integrate a plurality of functions such as 5G wireless communication, equipment identification, data acquisition, personnel activity state detection and the like, adopt an IEEE802.15.4 communication protocol, and have mature and stable technology. The method has the technical advantages of low transmitting power, high positioning accuracy (2-5 m), wide wireless coverage range (500 m), good communication confidentiality and the like.
In the embodiment, as shown in fig. 1, on-site video monitoring network architecture for early warning of abnormal behaviors of people is characterized in that on one side of a monitoring center, an uninterruptible power supply UPS 101 supplies power to a ground central station host 102, a rack-mounted video server performs machine learning and early warning management and control 103, the rack-mounted video server is connected to a bus through a switch 106, and a matrix 104 is connected with a hard disk video recorder 105 to record video image archive data of high-risk people; on the front terminal system side, the video/audio optical transceiver 109, and the transmission signal are connected to the camera gun 110, the camera 111, and the audio speaker 12 through the switch 108, and the warning information is controlled by the warning network 113 and the result is transmitted to the shared network. The user terminal may include a mobile phone, a laptop, a palmtop, a smart audio, a wearable device (e.g., a smart watch, a smart bracelet, etc.), and the like.
Example 2
The embodiment provides a method for early warning abnormal behaviors of people based on video data machine learning, which specifically includes the following steps as shown in fig. 2:
201. personnel pattern recognition in surveillance video images
The monitoring video image is subjected to feature extraction, and feature information of abnormal behaviors of the human body, including information of appearance clothes, body state, gait, pace, facial expression, mental state and the like, is effectively captured. The system understands the behavior and analyzes and identifies the motion pattern of the human body and describes the motion pattern.
202. Ghost data collection and reasoning
Through the experimental design process, whether ghost data in the forms of non-random missing data, working data and the like exist is checked from the data. If so, the structural model is used for verification, and a statistical machine learning method and dual robustness are used for reasoning.
The reasoning process for the data can be performed by using the existing model, and details are not described here.
203. Abnormal behavior reasoning based on machine learning
A method for analyzing and extracting key semantic regions from an image. Designing a tensor neural network to analyze and extract key semantic information in the video image, and decomposing the complete image into a plurality of image blocks according to different semantic information. Each image block expresses a semantic information, so that the human behavior space information can be expressed more carefully and pertinently. And jointly enhancing the key semantic block multiframe of the interframes by using a tensor convolutional neural network.
204. Determination of whether behavior is abnormal
And continuously optimizing through machine learning, and judging whether the behavior of the personnel is abnormal through modeling, system development and the like. And the abnormal behavior monitoring and early warning support system of the video image data is realized through the judgment of the abnormal behaviors with negative and abnormal rising.
205. Early warning decision support control panel
If the person has abnormal behaviors, pushing information for reminding, and realizing monitoring of abnormal behaviors of the person, early warning support and visualization.
Example 3
The embodiment provides a more specific implementation manner on the basis of the embodiment 2, and as shown in fig. 3, the warning method in the embodiment includes the following steps:
starting 301; the system uploads the surveillance video as metadata 303 to the system using the video image data intelligent collection process 302;
data cleaning 304, such as extracting, converting, and loading (ETL), is performed on the input data, which is mainly to clean low-quality video images in the source data and eliminate data that does not meet the specification. Missing data processing, similar repeated object detection, abnormal data processing, logic error detection, inconsistent data processing and the like are realized;
the video image data after cleaning is preprocessed 306, which includes two aspects:
for the blurred video image 307, the system adopts a frame quality enhancement method to improve the video definition;
analyzing and identifying 308 the motion pattern of the human body by using an algorithm;
classifying the preprocessed data, and assigning a classification label 309 to each type;
the multi-modal signature encoding 311 performs spatio-temporal desensitization data correlation 312 through systematic personnel identification; combining with face identification, using semi-structural data, such as videos and corresponding marks recorded by a sensor, and other life body, physiological activity and 3D behavior mark data, and a cascade personnel monitoring video image, three-dimensional posture human body moving image and other abnormal behavior case libraries 310, searching corresponding behavior specifications and monitoring information, realizing an early warning function module, and providing timely high-quality abnormal behavior risk management recommendation information;
these codes are subjected to feature engineering 314 to remove redundant features in the data and select features through information gain as inputs for use by the model and algorithm. The method mainly comprises two tasks: automatically selecting an algorithm; the expert participates in selecting the common feature set; the system understands the behaviors and analyzes and identifies the motion mode of the human body and describes the motion mode; meanwhile, feature extraction is carried out, and feature information of abnormal behaviors of the human body, including information of appearance clothes, body state, gait, pace, facial expression, mental state and the like, is effectively captured; dividing data into three types of normal, depression and emotional upsurge, taking characteristic data extracted from video images as condition attributes, marking the categories as decision attributes, and storing the decision attributes into a medium as a training case set;
acquiring the ghost data 313 through experimental design, and reasoning the ghost data through methods such as Fourier transform, Hadamard matrix and hidden variables;
the characteristics are mainly embodied in three aspects: basic information such as mental appearance, motion state and face data, and therefore three corresponding models are constructed: a mental state evaluation model 317, a walking gait recognition model 318 and a face data matching model 319. Through processes of knowledge discovery, optimization prediction, data analysis and the like; analyzing and responding to the video image through relevant principles and methods such as image identification, pattern identification, tensor neural network learning and the like; the trained learning algorithm can provide feedback information, improve parameter learning and model adjustment processes, and analyze and extract key semantic regions from images, a neural network is constructed to perform correlation calculation on key semantic regions between frames, and regions with consistent semantic information between frames are connected in series to obtain key semantic region chains between frames, so that the consistency of semantic information is ensured, time domain change information of human behaviors is expressed more pertinently, and the whole network utilizes global and local residual error structures to reduce training difficulty. Through continuous optimization of machine learning, modeling, system development and the like, a support system for monitoring and early warning of abnormal behaviors of people of video image data of an intelligent park is realized, evaluation indexes are provided by using a receiver operating characteristic Curve (ROC) and an Area Under the Curve (AUC), the model performance is comprehensively evaluated, and a prediction model and an algorithm are further feedback-controlled;
judging whether the human behavior is abnormal 320, if so, adding the information into an early warning database by the system and providing prompt information for early warning levels, verifying the human behavior identification method by using a related data set according to the abnormal behavior judgment accuracy of personnel, calculating the average value of the identification accuracy of various human behaviors as the result of human behavior identification, wherein the identification accuracy of human behavior is the percentage of the number of video images which can be correctly identified in certain behaviors to the total number of video images of the behaviors, and the alpha of certain behaviorsiAccuracy rate HiThe calculation is as follows:
in the formula, the condition is alphaiRepresenting rows of the actual i classCondition beta isiRepresenting behavior identified by the algorithm as class i; operator I represents the number of video frames which meet the calculation conditions;
the face data will be matched 321 with the face database of the high risk group of table 2, if it belongs to a high risk individual, the system will associate the personal information 324. Through the stages of lookup table processing, file association and dynamic matching, basic common sense logic judgment, field common sense judgment, data correlation verification and the like, early warning grades are divided for individuals by combining other abnormal information, and the early warning grades are added into an early warning database 322;
the system can set early warning parameters, when the real-time data exceeds the early warning range, the system can autonomously judge and send out warning signals, and according to actual requirements, pop-up window prompt information 323 is arranged on a system interface and is connected with an acousto-optic alarm to realize acousto-optic early warning;
remote early warning and linkage management and control 325, which detects multiple paths of early warning signals, automatically starts various corresponding linkage devices when early warning occurs, switches videos to corresponding cameras, triggers automatic video recording, gives an alarm to a monitoring center through a network, and pops up an alarm information prompt at a client until the end 326;
the obtained abnormal data information is stored in an abnormal behavior case base, data recycling of video images is achieved, feedback control is conducted on individual behaviors 316, monitoring data of each monitoring point on high-risk personnel are displayed in real time, the real-time data are continuously mined, analyzed and displayed through abundant online analysis graphs, historical monitoring data of the high-risk personnel are displayed in a graphical display mode and assisted with information research and judgment, meanwhile, historical monitoring data of the high-risk personnel are displayed in a chart mode and the like, and a historical data report is output and printable.
Table 1 example of characteristic data portion extracted by video image data and associated data
The method comprises the steps that various items of data in a table 1 are scored and assigned with weights according to actual conditions, when a weighted total score value exceeds a threshold value, a person is judged to be abnormal, in the detection process, the correlation between two images is calculated through Euclidean distance, if the person is normal but the emotion and gait change violently, for example, the facial expression changes from excitement to aversion and the pace speed changes from extremely slow to extremely fast, the data acquisition equipment needs to be enabled to perform enhanced tracking when the person is normal, and if the person continues to change extremely, the person is classified as an abnormal person and is notified to the related person.
Example 4
In this embodiment, on the basis of embodiment 1, an implementation manner of a human abnormal behavior early warning system based on video data machine learning is provided, as shown in fig. 4, a front end subsystem corresponds to a video monitoring device, a server corresponds to an abnormal behavior prediction device in a monitoring center, a control panel terminal corresponds to a display in the monitoring system, and an interaction manner of the video monitoring device, the server, and the control panel terminal includes the following steps:
transmitting a field monitoring video 401 through a video monitoring device, and then processing the obtained video image data 402;
obtaining a multi-modal feature coding frame 403, and performing feature engineering processing 404 to extract features in the multi-modal feature coding frame;
ghost data 405 are designed and constructed for the characteristics, and abnormal behavior reasoning is carried out on the data by combining a machine learning reinforcement learning model 406;
judging whether the behavior in the video image data belongs to an abnormal behavior 407;
classifying 408 the obtained abnormal behavior;
sending 409 the abnormal behavior information to a control panel terminal;
the early warning information 410 is issued by the control panel terminal.
As shown in fig. 5, in the abnormal behavior prediction apparatus in this embodiment, the video monitoring device acquires an image through the acquisition unit 501 and sends the image to the calculation unit for analysis, after the calculation unit 602 receives a signal, the control unit 603 performs design construction and controls an inference process on image operation, and after the calculation unit finishes working, the result is sent to the output unit 604.
Example 5
The embodiment provides a computer device for early warning of abnormal behaviors of people based on video data machine learning, which comprises a memory and a processor, wherein the memory is used for storing a method for early warning of abnormal behaviors of people, and whether abnormal behaviors exist in people in a monitor or not is predicted by running the method stored in the memory through the processor.
As shown in fig. 6, the processor 601 analyzes the image to transmit effective information, the memory 602 is responsible for storing data such as image, the communication interface 603 refers to the interface between the central processor and the standard communication subsystem and the bus 604, and the display 605 displays images.
The invention provides a method, a device, computer equipment and a medium for the personnel abnormal behavior early warning of an intelligent park based on a monitoring video image, provides a highly integrated information system function with an open architecture, meets the early warning requirements of different levels by flexibly configuring monitoring parameters, can play a positive promoting role for the human behavior recognition technology in the video image on the intellectualization of the fields of intelligent security, abnormal behavior prediction and the like, and can utilize a tensor neural network to obtain human behavior expression characteristics for human behavior recognition in the prior art. And designing and constructing human behavior expression characteristics by using image visual information, carrying out human behavior identification, combining a tensor neural network with manually constructed characteristics, carrying out human behavior identification and the like. The human body feature recognition algorithm is adopted to detect, analyze, recognize and early warn the features of the pedestrian target, meanwhile, the machine vision image perception technology is adopted to collect human body movement and facial features in a machine vision image scene, and the automatic recognition of the human body target and the prompt of early warning information are completed through the machine learning algorithm.
It will be understood by those skilled in the art that all or part of the processes in the method or technical solution for implementing the embodiments of the present invention may be implemented by hardware related to computer program instructions, and the program is stored in a computer readable storage medium, and when the program is executed, the processes of the embodiments of the method described above can be implemented. The storage medium may be an optical disc, a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. The method for early warning the abnormal behavior of the personnel based on the video data machine learning is characterized by comprising the following steps:
s1, acquiring video image data and cleaning the video image data;
s2, preprocessing the video image data, including removing information related to privacy in the data and adopting a frame quality enhancement method for the fuzzy video to improve the video definition;
s3, identifying the human motion mode, setting a classification label for each type, and performing multi-mode feature coding;
s4, performing feature engineering processing on the multi-mode codes, namely removing redundant features of the data center and selecting the features through information gain;
s5, constructing a tensor neural network to track people appearing in the video, carrying out gait detection based on a gait energy map according to tracked task contours, carrying out skin exposure detection on tasks in the video, taking the exposure degree as an appearance clothing index of the people, calculating video interframe images for correlation by integrating the indexes, and carrying out behavior abnormity judgment on the people according to the correlation calculation result; s6, early warning decision support is constructed, namely if the dynamic information of similarity and difference between the inter-frame images is acquired to show that the behavior of the shot person is abnormal, the shot person is informed to deal with the dynamic information.
2. The video data machine learning-based personnel abnormal behavior early warning method as claimed in claim 1, wherein after the characteristic engineering processing is performed in step S3, the data is detected, and if non-random missing data and working data exist, the data is inferred by a statistical machine learning method to obtain complete data of the data.
3. The video data machine learning-based people abnormal behavior early warning method as claimed in claim 1, wherein the multi-modal feature coding specifically comprises the following processes: using a 5-dimensional tensor structure for the acquired image, and encoding video data through (samples, frames, height, width, channels), wherein the 5-dimensional tensor data respectively represent samples, frames, height, width and channels of the video data; the acquired video image is subjected to a 4-dimensional tensor structure, and video data are coded through (samples, heights, widths, channels), so that 4-dimensional tensor data of samples, heights, widths and channels of the video data are respectively represented.
4. The video data machine learning-based person abnormal behavior early warning method according to claim 1, wherein tracking persons appearing in the video comprises:
taking the multi-modal coded image as the input of a tensor neural network;
the tensor neural network comprises human posture recognition, pedestrian detection and foreground and background separation;
acquiring a posture key sequence point of a human body in an image through human body posture identification;
detecting a person through pedestrian detection, segmenting a task from an image, and acquiring a posture outline sequence of the person by combining the separation of a foreground and a background;
and tracking and identifying the person through the acquired pose key sequence points and the pose outline sequence.
5. The method as claimed in claim 1, wherein the task in the video is detected for skin exposure as an indicator of the appearance of the person, that is, the skin area of the person in the image is obtained based on the elliptical skin model, and the ratio of the exposed skin area to the outline of the person is used as the indicator of the appearance of the person.
6. The video data machine learning-based personnel abnormal behavior early warning method as claimed in claim 1, wherein a sliding window with the length of N frames and the step length of t is set after gait detection and appearance clothing indexes are performed, the gait detection and appearance clothing indexes are valued through the sliding window, and the average value of the data of each window is taken as the data of correlation calculation.
7. The method for early warning of abnormal behaviors of people based on machine learning of video data according to claim 5 or 6, wherein the correlation between two images is calculated by Euclidean distance when difference judgment is performed.
8. The video data machine learning-based personnel abnormal behavior early warning method as claimed in claim 1, wherein when the behavior abnormality of the personnel is judged, if the similarity of two images extracted in the S frame is lower than a certain threshold value, the task abnormality is judged.
9. The system is characterized by comprising a front terminal system, a transmission network and a monitoring center, wherein the front terminal system comprises various monitoring devices, the front terminal system transmits information collected by the monitoring devices to the monitoring center through the transmission network, the monitoring center comprises an abnormal behavior prediction device, the device predicts whether a shot person is abnormal through the collected information, if so, the monitoring center gives an alarm and projects the monitoring device capable of shooting the person onto a monitoring center display for real-time monitoring.
10. The computer equipment for early warning of abnormal behaviors of people based on video data machine learning is characterized by comprising a memory and a processor, wherein the memory is used for storing any one of the method for early warning of abnormal behaviors of people as claimed in claims 1-8, and the processor is used for operating the method stored in the memory to predict whether the behaviors of people in a monitor are abnormal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111211548.7A CN113850229B (en) | 2021-10-18 | 2021-10-18 | Personnel abnormal behavior early warning method and system based on video data machine learning and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111211548.7A CN113850229B (en) | 2021-10-18 | 2021-10-18 | Personnel abnormal behavior early warning method and system based on video data machine learning and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113850229A true CN113850229A (en) | 2021-12-28 |
| CN113850229B CN113850229B (en) | 2023-12-22 |
Family
ID=78978708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111211548.7A Active CN113850229B (en) | 2021-10-18 | 2021-10-18 | Personnel abnormal behavior early warning method and system based on video data machine learning and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113850229B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218992A (en) * | 2021-12-29 | 2022-03-22 | 重庆紫光华山智安科技有限公司 | Abnormal object detection method and related device |
| CN115410324A (en) * | 2022-10-28 | 2022-11-29 | 山东世拓房车集团有限公司 | Car as a house night security system and method based on artificial intelligence |
| CN115883779A (en) * | 2022-10-13 | 2023-03-31 | 湖北公众信息产业有限责任公司 | Smart park information safety management system based on big data |
| CN116935286A (en) * | 2023-08-03 | 2023-10-24 | 广州城市职业学院 | Short video identification system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761516A (en) * | 2014-02-14 | 2014-04-30 | 重庆科技学院 | ATM abnormal human face detection method based on video monitoring |
| CN106919921A (en) * | 2017-03-06 | 2017-07-04 | 重庆邮电大学 | With reference to sub-space learning and the gait recognition method and system of tensor neutral net |
| CN109255312A (en) * | 2018-08-30 | 2019-01-22 | 罗普特（厦门）科技集团有限公司 | A kind of abnormal dressing detection method and device based on appearance features |
| CN109819208A (en) * | 2019-01-02 | 2019-05-28 | 江苏警官学院 | A kind of dense population security monitoring management method based on artificial intelligence dynamic monitoring |
| CN110443109A (en) * | 2019-06-11 | 2019-11-12 | 万翼科技有限公司 | Abnormal behaviour monitor processing method, device, computer equipment and storage medium |
| CN111353343A (en) * | 2018-12-21 | 2020-06-30 | 国家电网有限公司客户服务中心 | Business hall service standard quality inspection method based on video monitoring |
| CN113239759A (en) * | 2021-04-29 | 2021-08-10 | 国网江苏省电力有限公司苏州供电分公司 | Power grid operation field violation identification method |
-
2021
- 2021-10-18 CN CN202111211548.7A patent/CN113850229B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761516A (en) * | 2014-02-14 | 2014-04-30 | 重庆科技学院 | ATM abnormal human face detection method based on video monitoring |
| CN106919921A (en) * | 2017-03-06 | 2017-07-04 | 重庆邮电大学 | With reference to sub-space learning and the gait recognition method and system of tensor neutral net |
| CN109255312A (en) * | 2018-08-30 | 2019-01-22 | 罗普特（厦门）科技集团有限公司 | A kind of abnormal dressing detection method and device based on appearance features |
| CN111353343A (en) * | 2018-12-21 | 2020-06-30 | 国家电网有限公司客户服务中心 | Business hall service standard quality inspection method based on video monitoring |
| CN109819208A (en) * | 2019-01-02 | 2019-05-28 | 江苏警官学院 | A kind of dense population security monitoring management method based on artificial intelligence dynamic monitoring |
| CN110443109A (en) * | 2019-06-11 | 2019-11-12 | 万翼科技有限公司 | Abnormal behaviour monitor processing method, device, computer equipment and storage medium |
| CN113239759A (en) * | 2021-04-29 | 2021-08-10 | 国网江苏省电力有限公司苏州供电分公司 | Power grid operation field violation identification method |
Non-Patent Citations (1)
| Title |
|---|
| JINHUIWANG等: ""Anomaly Detection with Tensor Networks"", 《ARXIV》, pages 1 - 12 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218992A (en) * | 2021-12-29 | 2022-03-22 | 重庆紫光华山智安科技有限公司 | Abnormal object detection method and related device |
| CN114218992B (en) * | 2021-12-29 | 2023-09-08 | 重庆紫光华山智安科技有限公司 | Abnormal object detection method and related device |
| CN115883779A (en) * | 2022-10-13 | 2023-03-31 | 湖北公众信息产业有限责任公司 | Smart park information safety management system based on big data |
| CN115410324A (en) * | 2022-10-28 | 2022-11-29 | 山东世拓房车集团有限公司 | Car as a house night security system and method based on artificial intelligence |
| CN116935286A (en) * | 2023-08-03 | 2023-10-24 | 广州城市职业学院 | Short video identification system |
| CN116935286B (en) * | 2023-08-03 | 2024-01-09 | 广州城市职业学院 | Short video identification system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113850229B (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113850229B (en) | Personnel abnormal behavior early warning method and system based on video data machine learning and computer equipment | |
| CN110674772B (en) | Intelligent safety control auxiliary system and method for electric power operation site | |
| US11195067B2 (en) | Systems and methods for machine learning-based site-specific threat modeling and threat detection | |
| Weng et al. | Driver drowsiness detection via a hierarchical temporal deep belief network | |
| JP6905850B2 (en) | Image processing system, imaging device, learning model creation method, information processing device | |
| Chackravarthy et al. | Intelligent crime anomaly detection in smart cities using deep learning | |
| JP5560397B2 (en) | Autonomous crime prevention alert system and autonomous crime prevention alert method | |
| CN106355154B (en) | Method for detecting frequent passing of people in surveillance video | |
| CN111738218B (en) | Human body abnormal behavior recognition system and method | |
| Chang et al. | A hybrid CNN and LSTM-based deep learning model for abnormal behavior detection | |
| US11935297B2 (en) | Item monitoring for doorbell cameras | |
| CN116416281A (en) | Grain depot AI video supervision and analysis method and system | |
| Wang et al. | Video anomaly detection method based on future frame prediction and attention mechanism | |
| Girdhar et al. | Incept_LSTM: Accession for human activity concession in automatic surveillance | |
| Manocha et al. | Video-assisted smart health monitoring for affliction determination based on fog analytics | |
| CN116862740A (en) | Intelligent prison management and control system based on Internet | |
| CN114429677A (en) | Coal mine scene operation behavior safety identification and assessment method and system | |
| Altowairqi et al. | A Review of the Recent Progress on Crowd Anomaly Detection | |
| CN113065500A (en) | Abnormal behavior control system for special actions | |
| Schuster et al. | Multi-cue learning and visualization of unusual events | |
| Krishnan et al. | Automatic Detection of Anomalies in Video Surveillance using Artificial Intelligence | |
| Nithesh et al. | Anomaly Detection in Surveillance Videos Using Deep Learning | |
| US20230360402A1 (en) | Video-based public safety incident prediction system and method therefor | |
| Ma et al. | Research on Person Re-identification Method Based on Data Link Technology | |
| Horowitz et al. | Interruptions detection in video conferences |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231128 Address after: 518000 No. 1, west side of 5 / F, building e, tefa information port building, No. 2 Kefeng Road, Nanshan District, Shenzhen, Guangdong Applicant after: Shenzhen Chengyu Information Technology Co.,Ltd. Address before: Room 801, 85 Kefeng Road, Huangpu District, Guangzhou City, Guangdong Province Applicant before: Guangzhou Dayu Chuangfu Technology Co.,Ltd. Effective date of registration: 20231128 Address after: Room 801, 85 Kefeng Road, Huangpu District, Guangzhou City, Guangdong Province Applicant after: Guangzhou Dayu Chuangfu Technology Co.,Ltd. Address before: 400065 Chongwen Road, Nanshan Street, Nanan District, Chongqing Applicant before: CHONGQING University OF POSTS AND TELECOMMUNICATIONS |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |