CN112257545A - Violation real-time monitoring and analyzing method and device and storage medium - Google Patents
Violation real-time monitoring and analyzing method and device and storage medium Download PDFInfo
- Publication number
- CN112257545A CN112257545A CN202011116520.0A CN202011116520A CN112257545A CN 112257545 A CN112257545 A CN 112257545A CN 202011116520 A CN202011116520 A CN 202011116520A CN 112257545 A CN112257545 A CN 112257545A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- training
- model
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 111
- 230000006399 behavior Effects 0.000 claims abstract description 54
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 230000002159 abnormal effect Effects 0.000 claims abstract description 11
- 238000012360 testing method Methods 0.000 claims description 44
- 238000012795 verification Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 12
- 238000010200 validation analysis Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000009471 action Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000391 smoking effect Effects 0.000 description 6
- 230000015654 memory Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000779 smoke Substances 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention provides a real-time monitoring and analyzing method for illegal behaviors, which comprises the steps of firstly collecting a plurality of pieces of public behavior video data and real scene video data as training data sets; preprocessing the training data set to obtain preprocessed data; performing off-line training on the preprocessed data to obtain an off-line training model; collecting real-time video stream data of a camera of a real scene; performing behavior recognition on the real-time video data through the offline training model, and judging whether the real-time video data contains an illegal behavior; and if the implementation video data contains the violation behaviors, outputting an abnormal warning signal. By utilizing the invention, the acquired video data can be intelligently analyzed in real time, and the missing report and the false report caused by manual large-screen monitoring are reduced.
Description
Technical Field
The invention relates to the field of behavior monitoring of videos, in particular to a method, a device and a storage medium for monitoring and analyzing violation behaviors in real time.
Background
At present, the illegal behaviors in a specific area can be monitored in real time, for example, the illegal behaviors are used for monitoring the illegal behaviors of soldiers, the public network mobile phone is strictly forbidden to be carried and used in combat readiness, training and exercise tasks, and the soldiers cannot smoke in public places and other places where smoking is forbidden. Inside military camps, the supervising personnel on duty monitors illegal behaviors in real time through a large screen, but the manual monitoring method has potential safety hazards and is easy to cause false reports and missing reports.
Disclosure of Invention
In view of the above disadvantages of the prior art, an object of the present invention is to provide a method for monitoring and analyzing violations in real time, which can monitor the violations in military camps in real time, find the violations in time, and take corresponding measures to reduce false reports and false reports caused by manual large-screen monitoring.
In order to achieve the above and other related objects, the present invention provides a method for real-time monitoring and analyzing violation, which comprises the following steps:
collecting a plurality of public behavior video data and real scene video data as a training data set;
preprocessing the training data set to obtain preprocessed data;
performing off-line training on the preprocessed data to obtain an off-line training model;
testing the obtained off-line training model, and recording a test result;
collecting real-time video stream data of a camera of a real scene;
performing behavior recognition on the real-time video stream data through the offline training model, and judging whether the real-time video stream data contains an illegal behavior; and if the real-time video stream data contains the violation behavior, outputting an abnormal warning signal.
In an embodiment of the present invention, the step of preprocessing the training data set includes: and screening the training data set, converting the video into image frame data, deleting redundant data and expanding the training data set.
In one embodiment of the invention, the set of preprocessed data includes a training set, a validation set, and a test set, the data of the training set is used for training, the data of the validation set is used for validation, and the data of the test set is used for testing.
In an embodiment of the present invention, the offline training model includes a deep residual network ResNet50, a time shift model TSM, and a non-local connection network model non-local.
In an embodiment of the present invention, the depth residual error network ResNet50 includes 4 block units, and the first block unit, the second block unit, the third block unit, and the fourth block unit respectively include 3, 4, 6, and 3 convolutional layers.
In an embodiment of the invention, the time-shift model TSM is applied to all convolutional layers in the block unit, and the non-locally connected network model non-local is applied to the 1 st, 3 rd convolutional layers of the second block unit and the 1 st, 3 rd, 5 th convolutional layers of the third block unit.
In an embodiment of the present invention, the off-line training model constructed is used to perform training and verification on the training set and the verification set to obtain a trained optimal model, then the trained optimal model is used to perform testing on the test set to obtain an image classification result, and the accuracy of the classification result is recorded.
In an embodiment of the present invention, the method for acquiring the camera real-time video stream data of the real scene includes: and converting the real-time video stream of the key area camera into picture frame data by adopting the real-time video stream of the key area camera.
In order to achieve the above and other related objects, the present invention further provides an illegal action real-time monitoring and analyzing device, which includes:
the data control unit comprises a data acquisition unit and a data processor, wherein the data acquisition unit is connected with the data processor and is used for acquiring a plurality of public behavior video data and real scene video data as training data sets; the data processor is used for preprocessing the training data to obtain preprocessed data;
the model debugging unit is connected with the data controller and comprises a training processing module and a testing processing module; the training processing module carries out off-line training on the preprocessed data to obtain an off-line training model; the test processing module tests the obtained off-line training model and records a test result;
and the real-time behavior monitoring and analyzing unit is connected with the model debugging unit and is used for acquiring real-time video stream data of a camera in a real scene, performing behavior recognition on the real-time video stream data through the trained model, judging whether the real-time video stream data contains illegal behaviors or not, and outputting an abnormal warning signal if the real-time video stream data contains the illegal behaviors.
To achieve the above and other related objects, the present invention further provides a computer-readable storage medium storing a computer program, which is used to execute a method for performing real-time monitoring and analysis of violations, the method being implemented as described in the above embodiments.
As described above, the method for monitoring and analyzing the violation in real time according to the embodiment of the present invention applies the time shift model TSM and the non-local connection network non-local to the convolution layer of the deep residual error network ResNet50, so that the accuracy of image classification of the model can be improved, the model has a small operation calculation amount and a low calculation complexity, and can monitor the behavior in military camps in real time, find the violation in time, and take corresponding measures to reduce false negative and false positive caused by manual large-screen monitoring.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for real-time monitoring and analyzing an illegal action according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an apparatus for monitoring and analyzing an illegal action in real time according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data controller according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a model debugging unit according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For example, military is used as an important special place at national level, and on one hand, the regional characteristic of high sensitivity determines the strong requirement of security level control; on the other hand, the interior of the military also has strict discipline and management requirements. This makes it a trend to build highly intelligent monitoring systems. The application of intelligent video monitoring in the military forces prompts the military safety management to really realize the conversion from manual precaution to technical precaution. Machine learning is a multi-field cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. The intelligent video analysis system improves the effectiveness and the continuity of monitoring, has quicker response time and stronger data retrieval and analysis functions, greatly improves the monitoring capability, more effectively assists safety personnel to deal with attacks and handle emergencies, assists management personnel to carry out scientific and quantifiable management, and assists workers to easily maintain a large-scale security system.
In some embodiments, feature fusion between different frames can be achieved by a video time domain information fusion method of a TSM (Temporal Shift Module) in a time dimension feature displacement manner without adding additional parameters. The TSM uses time displacement to improve the network video understanding capability, a part of characteristic diagram channels are displaced forward by one step in a time dimension, a part of channels are displaced backward by one step in the time dimension, and the displaced vacancy is filled with zero, so that the context interaction in the time dimension is introduced into the characteristic diagram.
In some embodiments, the target feature point may be calculated through a non-local Neural Networks (non-local Networks), and information of surrounding feature points may be combined at the same time, for example, the target feature point may be calculated in a time dimension or a space dimension. By defining a correlation function between the output location and all input locations, creating an operation with global correlation properties, each location on the output signature is affected by the data of all locations on the input signature.
In some embodiments, ResNet50 is a more balanced choice of performance and resource consumption in the ResNet (Deep residual network) family of networks.
In different embodiments, the invention provides a real-time monitoring and analyzing method for illegal behaviors, which comprises the steps of firstly collecting a plurality of pieces of public behavior video data and real scene video data as training data sets; preprocessing the training data set to obtain preprocessed data; performing off-line training on the preprocessed data to obtain an off-line training model; collecting real-time video stream data of a camera of a real scene; performing behavior recognition on the real-time video data through the offline training model, and judging whether the real-time video data contains an illegal behavior; and if the implementation video data contains the violation behaviors, outputting an abnormal warning signal. By using the invention, abnormal behaviors in military camps can be monitored in real time.
Referring to fig. 1, an embodiment of the present invention provides a method for monitoring and analyzing an illegal action in real time, including:
s101: and acquiring a plurality of public behavior video data and real scene video data as a training data set.
It should be noted that the public behavior video data may be obtained from the internet, for example, a video data set such as HMDB51, HOLLYWOOD2, or the like may be obtained. The real scene video data can be, for example, a video clip of a real violation in military camp, and can be, for example, a video clip of a telephone call, a smoke, and the like. And extracting the video clips for making a call and smoking from the public behavior video data and the real scene video data for classification.
S102: and preprocessing the training data set to obtain preprocessed data.
It should be noted that the preprocessing may include processes of screening the training data set, converting video into image frame data, deleting redundant data, expanding the training data set, and the like. For example, the opencv library of Python may be utilized to process video data and convert the video data into image frame data. For example, histogram similarity may be used to calculate two image frames before and after the same video data segmentData framei-1And frameiS, said S being a framei-framei-1If the similarity S is smaller than the value X, for example X may range from 0.6 to 0.8, for example 0.7, then the image frame data frame is obtainedi-1The image frame data frame being the end of the current segmentiAs the starting point for the next segment. When a section of complete video data image is divided into a plurality of sub-image frame data segments, deleting the segments with the frame number less than 7 in the sub-image frame data segments, wherein the segments with the frame number less than 7 are regarded as noise segments, and each sub-image frame data segment sub _ frames reserved is used as a newly added training data set, so that the more abundant the training data set is, the accuracy of the real-time monitoring analysis method is improved. Dividing the obtained preprocessing data into a training set, a verification set and a test set, wherein the proportion of the training set, the verification set and the test set can be 8:1:1 or 6:2:2, and the like, the data of the training set is used for training, the data of the verification set is used for verifying, the data of the test set is used for testing, and the larger the proportion of the training set is, the more the training data is, and the accuracy of the real-time monitoring and analyzing method can be improved.
S103: and performing off-line training on the preprocessed data to obtain an off-line training model.
It should be noted that the infrastructure of the offline training model may be a ResNet50 model, the ResNet50 model may be formed by a plurality of block units, the block units are residual error units, the ResNet50 model may include 4 block units, for example, and the first block unit, the second block unit, the third block unit, and the fourth block unit may include 3 convolutional layers, 4 convolutional layers, 6 convolutional layers, and 3 convolutional layers, for example. The convolutional layer is composed of a plurality of convolutional units, parameters of each convolutional unit are obtained through optimization of a back propagation algorithm, the purpose of convolutional operation is to extract different input features, the convolutional layer at the first layer can only extract some low-level features such as edges, lines, angles and other levels, and more layers of networks can iteratively extract more complex features from the low-level features. Applying the TSM model and the non-local model to the convolutional layer of the ResNet50 model can improve the accuracy of image classification, and the TSM model and the non-local model will have different effects due to different numbers and insertion positions. For example, in one embodiment of the present invention, the TSM model may be applied to all convolutional layers in the block unit, and the non-local model may be applied to, for example, the 1 st and 3 rd convolutional layers of the second block unit and the 1 st, 3 rd and 5 th convolutional layers of the third block unit, so as to further improve the accuracy of the offline training model. The offline training model mainly discriminates two types of violations, namely smoking and calling, for example, the output parameter of the full-connection classification layer of the ResNet50 model can be set to 2, for example.
Referring to fig. 1, in an embodiment of the present invention, a trained optimal image classification model is obtained by training and verifying a training set and a verification set using a constructed model structure, then an image classification result is obtained by testing the trained optimal image classification model on a test set, and the classification accuracy of the classification result is recorded, so as to complete a complete training, verification and testing, and the same model parameters are used in the complete training, verification and testing. For example, the data of the training set is trained by using a constructed ResNet50+ TSM + Non-local model, the training time can be 20-35 times, for example, 30 times, and after each training, the data of the verification set is used for verification and the prediction accuracy of the model is recorded. The number of training sets is for example 1000, the number of validation sets is for example 125, and if the data of for example 100 validation sets is correct at the time of validation, the prediction accuracy of the model is for example 80%. And if the prediction accuracy of the current training model is greater than that of the last training model, saving the current model. In order to compare the accuracy of different models, the constructed ResNet50+ TSM model and ResNet50+ Non-Local model are also trained and verified by using the same training data, and the prediction accuracy is recorded.
S104: and testing the obtained off-line training model, and recording a test result.
It should be noted that, the three models are tested by using the data of the test set, for example, the data of the test set is input into the offline training model, the offline training model outputs a judgment result, and whether the output judgment result is consistent with the real label of the test set is observed. For example, the accuracy of the ResNet50+ TSM + Non-Local model, ResNet50+ TSM model, ResNet50+ Non-Local model is 91.5%, 90.0%, 89.7%, respectively.
S105: and acquiring real-time video stream data of a camera of a real scene.
It should be noted that the real-time video stream data may be obtained by camera monitoring, for example, a Python opencv library is used to obtain a real-time video stream of a key area camera in military camps, for example, the real-time video stream may be an RTSP video stream, and is converted into picture frame data, for example, the picture frame data may be divided into a plurality of groups, for example, 8 picture frames may be a group.
S106: and performing behavior recognition on the real-time video stream data through the offline training model, judging whether the real-time video stream data contains illegal behaviors, and outputting an abnormal warning signal if the real-time video stream data contains the illegal behaviors.
It should be noted that, the optimal offline training model is selected, the picture frame group data is input into the optimal offline training model for recognition and analysis, and whether picture frame data of an illegal action is included is judged. And if the picture frame data of the illegal behaviors of calling and smoking are identified by the optimal offline training model, outputting an abnormal warning signal and sending the abnormal warning signal to an operator on duty to take measures in time.
Referring to fig. 2 and fig. 3, in an embodiment, the present invention further provides an apparatus for real-time monitoring and analyzing an illegal action, including: the data control unit 1 comprises a data collector 11 and a data processor 12, the data collector 11 is connected with the data processor 12, the data collector 11 is used for collecting training data, and the training data comprises public behavior video data sets on a network and real scene violation video data sets in military camps, such as video clips for smoking and calling. The data processor 12 is configured to perform preprocessing on the training data to obtain preprocessed data, where the preprocessing may be, for example, data screening, video conversion into image frames, redundant data deletion, training data set expansion, and the like. The obtained preprocessing data can be divided into a training set, a verification set and a test set, the proportion of the training set, the verification set and the test set can be 8:1:1 or 6:2:2, and the like, and the larger the proportion of the training set is, the more training data is, and the accuracy of the real-time monitoring and analyzing device can be improved. For example, the data control unit 1 may be provided in a CPU server.
Referring to fig. 2 and 4, the real-time monitoring and analyzing apparatus further includes a model debugging unit 2, the model debugging unit 2 is connected to the data control unit 1, and the model debugging unit 2 includes a training processing module 21 and a testing processing module 22. For example, the function of the model debugging unit 2 may be written in Python programming language, the GPU server operates, the model may be, for example, a ResNet50+ TSM + Non-local model, the ResNet50 model may be composed of a plurality of block units, and the ResNet50 model may include, for example, 4 block units, and for example, the first block unit, the second block unit, the third block unit, and the fourth block unit may respectively include 3, 4, 6, and 3 convolutional layers. For example, the TSM model may be applied to all of the convolutional layers in the block unit, and the non-local model may be applied to, for example, the 1 st, 3 rd convolutional layers of the second block unit and the 1 st, 3 rd, 5 th convolutional layers of the third block unit, and thus applied for the purpose of improving the accuracy of the model. The model mainly discriminates two types of violations, namely smoking and calling, for example, the output parameter of the full-connection classification layer of the ResNet50 model can be set to 2, for example. The training processing module 21 trains the preprocessed data, and the testing processing module 22 tests the trained model, for example, training and verifying on a training set and a verification set by using a constructed model structure to obtain a trained optimal model, then testing on a testing set by using the trained optimal model to obtain an image classification result, and recording the classification accuracy of the classification result, i.e., completing a complete training, verification and testing, wherein the same model parameters are used in the complete training, verification and testing.
Referring to fig. 2, the real-time monitoring and analyzing apparatus further includes a real-time behavior monitoring and analyzing unit 3, the real-time behavior monitoring and analyzing unit 3 is connected to the model debugging unit 2, and the real-time monitoring and analyzing unit 3 is configured to collect real-time video stream data of a camera in a real scene, for example, the real-time video data may be obtained through monitoring of the camera, perform behavior recognition on the real-time video stream data through the trained model, determine whether the real-time video stream data includes an illegal behavior, and output an abnormal warning signal if the real-time video stream data includes the illegal behavior.
Referring to fig. 5, in an embodiment, the present invention further provides a computer-readable storage medium 4 storing a computer program, where the computer program is used to execute steps of implementing any one of the methods for real-time monitoring and analyzing an illegal action provided by the embodiments of the present invention. Embodiments of the present invention provide computer-readable storage media including, but not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magnetic-optical disks, ROMs (Read-Only memories), RAMs (random access memories), EPROMs (Erasable Programmable Read-Only memories), EEPROMs (Electrically Erasable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a readable storage medium includes any medium that stores or transmits information in a form readable by a device (e.g., a computer).
In summary, the method for monitoring and analyzing the violation in real time provided by the present invention collects a plurality of public behavior video data and real scene video data as a training data set; preprocessing the training data set to obtain preprocessed data; performing off-line training on the preprocessed data to obtain an off-line training model; collecting real-time video data of a real scene; performing behavior recognition on the real-time video data through the offline training model, and judging whether the real-time video data contains an illegal behavior; and if the implementation video data contains the violation behaviors, outputting an abnormal warning signal. The invention aims to intelligently analyze the acquired video data in real time, discover the violation of military personnel in time and reduce the missing report and the false report caused by manual large-screen monitoring.
The systems and methods have been described herein in general terms as the details aid in understanding the invention. Furthermore, various specific details have been given to provide a general understanding of the embodiments of the invention. One skilled in the relevant art will recognize, however, that an embodiment of the invention can be practiced without one or more of the specific details, or with other apparatus, systems, assemblies, methods, components, materials, parts, and/or the like. In other instances, well-known structures, materials, and/or operations are not specifically shown or described in detail to avoid obscuring aspects of embodiments of the invention.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (10)
1. A real-time monitoring and analyzing method for violation behaviors is characterized by comprising the following steps:
collecting a plurality of public behavior video data and real scene video data as a training data set;
preprocessing the training data set to obtain preprocessed data;
performing off-line training on the preprocessed data to obtain an off-line training model;
testing the obtained off-line training model, and recording a test result;
collecting real-time video stream data of a camera of a real scene;
performing behavior recognition on the real-time video stream data through the offline training model, and judging whether the real-time video stream data contains an illegal behavior; and if the real-time video stream data contains the violation behavior, outputting an abnormal warning signal.
2. The method according to claim 1, wherein the step of preprocessing the training data set comprises:
and screening the training data set, converting the video into image frame data, deleting redundant data and expanding the training data set.
3. The method according to claim 1, wherein the set of preprocessed data includes a training set, a validation set, and a test set, the training set data is used for training, the validation set data is used for validation, and the test set data is used for testing.
4. The method according to claim 1, wherein the offline training model comprises a deep residual error network ResNet50, a time shift model TSM, and a non-local connection network model non-local.
5. The method according to claim 4, wherein the deep residual error network ResNet50 includes 4 block units, and the first, second, third, and fourth block units include 3, 4, 6, and 3 convolutional layers, respectively.
6. The method of claim 5, wherein the TSM is applied to all convolutional layers in the block unit, and the non-local connection network model non-local is applied to the 1 st and 3 rd convolutional layers of the second block unit and the 1 st, 3 rd and 5 th convolutional layers of the third block unit.
7. The method for monitoring and analyzing the violation in real time according to claim 3, wherein the constructed offline training model is used for training and verifying on the training set and the verification set to obtain a trained optimal model, then the trained optimal model is used for testing on the test set to obtain an image classification result, and the accuracy of the classification result is recorded.
8. The method for monitoring and analyzing the violation real-time according to claim 1, wherein the method for collecting the real-time video stream data of the camera of the real scene comprises the following steps:
and converting the real-time video stream of the key area camera into picture frame data by adopting the real-time video stream of the key area camera.
9. A violation real-time monitoring and analyzing device is characterized by comprising:
the data control unit comprises a data acquisition unit and a data processor, wherein the data acquisition unit is connected with the data processor and is used for acquiring a plurality of public behavior video data and real scene video data as training data sets; the data processor is used for preprocessing the training data to obtain preprocessed data;
the model debugging unit is connected with the data control unit and comprises a training processing module and a testing processing module; the training processing module carries out off-line training on the preprocessed data to obtain an off-line training model; the test processing module tests the obtained off-line training model and records a test result;
and the real-time behavior monitoring and analyzing unit is connected with the model debugging unit and is used for acquiring real-time video stream data of a camera in a real scene, performing behavior recognition on the real-time video stream data through the trained model, judging whether the real-time video stream data contains illegal behaviors or not, and outputting an abnormal warning signal if the real-time video stream data contains the illegal behaviors.
10. A computer-readable storage medium storing a computer program for execution by a processor to implement a method of real-time violation monitoring and analysis according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011116520.0A CN112257545A (en) | 2020-10-19 | 2020-10-19 | Violation real-time monitoring and analyzing method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011116520.0A CN112257545A (en) | 2020-10-19 | 2020-10-19 | Violation real-time monitoring and analyzing method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112257545A true CN112257545A (en) | 2021-01-22 |
Family
ID=74244616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011116520.0A Pending CN112257545A (en) | 2020-10-19 | 2020-10-19 | Violation real-time monitoring and analyzing method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257545A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114310488A (en) * | 2021-12-27 | 2022-04-12 | 深圳市玄羽科技有限公司 | Method for generating tool fracture detection model, detection method, device and medium |
| CN114913172A (en) * | 2022-07-13 | 2022-08-16 | 广东电网有限责任公司佛山供电局 | Method, system, equipment and medium for identifying manufacturing risk of cable middle head |
| CN117877219A (en) * | 2023-12-20 | 2024-04-12 | 山东方垠智能制造有限公司 | Illegal action alarm method, system, storage medium and equipment |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469383A (en) * | 2014-12-30 | 2016-04-06 | 北京大学深圳研究生院 | Wireless capsule endoscopy redundant image screening method based on multi-feature fusion |
| CN109034092A (en) * | 2018-08-09 | 2018-12-18 | 燕山大学 | Accident detection method for monitoring system |
| CN109711320A (en) * | 2018-12-24 | 2019-05-03 | 兴唐通信科技有限公司 | A kind of operator on duty's unlawful practice detection method and system |
| CN110084151A (en) * | 2019-04-10 | 2019-08-02 | 东南大学 | Video abnormal behaviour method of discrimination based on non-local network's deep learning |
| CN110263728A (en) * | 2019-06-24 | 2019-09-20 | 南京邮电大学 | Anomaly detection method based on improved pseudo- three-dimensional residual error neural network |
| CN110457984A (en) * | 2019-05-21 | 2019-11-15 | 电子科技大学 | Pedestrian's attribute recognition approach under monitoring scene based on ResNet-50 |
| CN110516529A (en) * | 2019-07-09 | 2019-11-29 | 杭州电子科技大学 | It is a kind of that detection method and system are fed based on deep learning image procossing |
| WO2019246008A1 (en) * | 2018-06-19 | 2019-12-26 | Honeywell International Inc. | Autonomous predictive real-time monitoring of faults in process and equipment |
| CN110717433A (en) * | 2019-09-30 | 2020-01-21 | 华中科技大学 | Deep learning-based traffic violation analysis method and device |
| CN111062278A (en) * | 2019-12-03 | 2020-04-24 | 西安工程大学 | Abnormal behavior identification method based on improved residual error network |
| CN111259785A (en) * | 2020-01-14 | 2020-06-09 | 电子科技大学 | Lip language identification method based on time offset residual error network |
| CN111325144A (en) * | 2020-02-19 | 2020-06-23 | 上海眼控科技股份有限公司 | Behavior detection method and apparatus, computer device and computer-readable storage medium |
| CN111369299A (en) * | 2020-03-11 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Method, device and equipment for identification and computer readable storage medium |
| CN111523566A (en) * | 2020-03-31 | 2020-08-11 | 易视腾科技股份有限公司 | Target video clip positioning method and device |
| CN111582073A (en) * | 2020-04-23 | 2020-08-25 | 浙江大学 | Transformer substation violation identification method based on ResNet101 characteristic pyramid |
| CN111601162A (en) * | 2020-06-08 | 2020-08-28 | 北京世纪好未来教育科技有限公司 | Video segmentation method and device and computer storage medium |
-
2020
- 2020-10-19 CN CN202011116520.0A patent/CN112257545A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469383A (en) * | 2014-12-30 | 2016-04-06 | 北京大学深圳研究生院 | Wireless capsule endoscopy redundant image screening method based on multi-feature fusion |
| WO2019246008A1 (en) * | 2018-06-19 | 2019-12-26 | Honeywell International Inc. | Autonomous predictive real-time monitoring of faults in process and equipment |
| CN109034092A (en) * | 2018-08-09 | 2018-12-18 | 燕山大学 | Accident detection method for monitoring system |
| CN109711320A (en) * | 2018-12-24 | 2019-05-03 | 兴唐通信科技有限公司 | A kind of operator on duty's unlawful practice detection method and system |
| CN110084151A (en) * | 2019-04-10 | 2019-08-02 | 东南大学 | Video abnormal behaviour method of discrimination based on non-local network's deep learning |
| CN110457984A (en) * | 2019-05-21 | 2019-11-15 | 电子科技大学 | Pedestrian's attribute recognition approach under monitoring scene based on ResNet-50 |
| CN110263728A (en) * | 2019-06-24 | 2019-09-20 | 南京邮电大学 | Anomaly detection method based on improved pseudo- three-dimensional residual error neural network |
| CN110516529A (en) * | 2019-07-09 | 2019-11-29 | 杭州电子科技大学 | It is a kind of that detection method and system are fed based on deep learning image procossing |
| CN110717433A (en) * | 2019-09-30 | 2020-01-21 | 华中科技大学 | Deep learning-based traffic violation analysis method and device |
| CN111062278A (en) * | 2019-12-03 | 2020-04-24 | 西安工程大学 | Abnormal behavior identification method based on improved residual error network |
| CN111259785A (en) * | 2020-01-14 | 2020-06-09 | 电子科技大学 | Lip language identification method based on time offset residual error network |
| CN111325144A (en) * | 2020-02-19 | 2020-06-23 | 上海眼控科技股份有限公司 | Behavior detection method and apparatus, computer device and computer-readable storage medium |
| CN111369299A (en) * | 2020-03-11 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Method, device and equipment for identification and computer readable storage medium |
| CN111523566A (en) * | 2020-03-31 | 2020-08-11 | 易视腾科技股份有限公司 | Target video clip positioning method and device |
| CN111582073A (en) * | 2020-04-23 | 2020-08-25 | 浙江大学 | Transformer substation violation identification method based on ResNet101 characteristic pyramid |
| CN111601162A (en) * | 2020-06-08 | 2020-08-28 | 北京世纪好未来教育科技有限公司 | Video segmentation method and device and computer storage medium |
Non-Patent Citations (6)
| Title |
|---|
| JI LIN等: "TSM: Temporal Shift Module for Efficient Video Understanding" * |
| JI LIN等: "TSM: Temporal Shift Module for Efficient Video Understanding", 《2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
| XIAOLONG WANG等: "Non-local Neural Networks" * |
| XIAOLONG WANG等: "Non-local Neural Networks", 《2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 余博思: "基于低秩逼近的视频序列中的异常事件检测", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 杨珂等: "LSCN:一种用于动作识别的长短时序关注网络", 《电子学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114310488A (en) * | 2021-12-27 | 2022-04-12 | 深圳市玄羽科技有限公司 | Method for generating tool fracture detection model, detection method, device and medium |
| CN114310488B (en) * | 2021-12-27 | 2023-10-27 | 深圳市玄羽科技有限公司 | Method for generating cutter fracture detection model, detection method, equipment and medium |
| CN114913172A (en) * | 2022-07-13 | 2022-08-16 | 广东电网有限责任公司佛山供电局 | Method, system, equipment and medium for identifying manufacturing risk of cable middle head |
| CN114913172B (en) * | 2022-07-13 | 2022-12-30 | 广东电网有限责任公司佛山供电局 | Method, system, equipment and medium for identifying manufacturing risk of cable middle head |
| CN117877219A (en) * | 2023-12-20 | 2024-04-12 | 山东方垠智能制造有限公司 | Illegal action alarm method, system, storage medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111741356B (en) | Quality inspection method, device and equipment for double-recording video and readable storage medium | |
| CN112257545A (en) | Violation real-time monitoring and analyzing method and device and storage medium | |
| Liu et al. | Generalized video anomaly event detection: Systematic taxonomy and comparison of deep models | |
| CN111369299B (en) | Identification method, device, equipment and computer readable storage medium | |
| KR102217253B1 (en) | Apparatus and method for analyzing behavior pattern | |
| CN112434178B (en) | Image classification method, device, electronic equipment and storage medium | |
| US11037604B2 (en) | Method for video investigation | |
| CN110008102B (en) | Server performance testing method and system based on intelligent video application | |
| CN112507860A (en) | Video annotation method, device, equipment and storage medium | |
| CN112380922B (en) | Method, device, computer equipment and storage medium for determining multiple video frames | |
| CN111353399A (en) | Tamper video detection method | |
| CN114758271A (en) | Video processing method, device, computer equipment and storage medium | |
| Kumar et al. | Crime activities prediction system in video surveillance by an optimized deep learning framework | |
| CN112686186A (en) | High-altitude parabolic recognition method based on deep learning and related components thereof | |
| CN116189063B (en) | Key frame optimization method and device for intelligent video monitoring | |
| CN110490079A (en) | Inspection data processing method, device, computer equipment and storage medium | |
| CN112272277B (en) | Voice adding method and device in nuclear power test and computer equipment | |
| CN114241401B (en) | Abnormality determination method, abnormality determination device, abnormality determination apparatus, abnormality determination medium, and abnormality determination product | |
| CN116248346A (en) | Smart city-oriented CPS network security situation awareness establishing method and system | |
| CN114882229A (en) | Target detection picture generation method, device, equipment and medium | |
| CN112364714B (en) | Face recognition method, device, computer equipment and storage medium | |
| CN111145558B (en) | Illegal behavior identification method based on high-point video monitoring | |
| CN114241400A (en) | Monitoring method and device of power grid system and computer readable storage medium | |
| CN118135377B (en) | Model deployment method, terminal side equipment and storage medium | |
| CN112380393B (en) | Ticket holding test method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Xu Guo Inventor after: Jiang Hancheng Inventor after: Xu Lei Inventor after: Zhang Xinxuan Inventor after: Wu Feng Inventor after: Yu Xiaohu Inventor after: Li Yangyang Inventor after: Xu Bin Inventor after: Xiong Zhongyuan Inventor after: Su Dan Inventor after: Zhu Zhenyu Inventor before: Xu Guo Inventor before: Li Yangyang Inventor before: Jiang Hancheng Inventor before: Xu Bin Inventor before: Su Dan Inventor before: Zhang Xinxuan Inventor before: Xiong Zhongyuan Inventor before: Zhu Zhenyu Inventor before: Xu Lei Inventor before: Yu Xiaohu |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210122 |