CN117237993B - Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment - Google Patents
Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN117237993B CN117237993B CN202311490614.8A CN202311490614A CN117237993B CN 117237993 B CN117237993 B CN 117237993B CN 202311490614 A CN202311490614 A CN 202311490614A CN 117237993 B CN117237993 B CN 117237993B
- Authority
- CN
- China
- Prior art keywords
- layer
- model
- module
- conv2d
- gbcsp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000003860 storage Methods 0.000 title claims abstract description 7
- 230000006399 behavior Effects 0.000 title claims description 33
- 238000001514 detection method Methods 0.000 claims abstract description 71
- 230000007246 mechanism Effects 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 230000004927 fusion Effects 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 17
- 238000012360 testing method Methods 0.000 claims description 16
- 238000011156 evaluation Methods 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 11
- 230000009471 action Effects 0.000 claims description 9
- 230000002457 bidirectional effect Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 2
- 238000007906 compression Methods 0.000 claims description 2
- 238000010008 shearing Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 7
- 230000009466 transformation Effects 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000000391 smoking effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The disclosure discloses a method, a device, a storage medium and electronic equipment for detecting operation site violations, wherein the method comprises the following steps: s100: collecting video stream data of an operation site; s200: preprocessing the video stream data; s300: constructing an illegal behavior detection model and training; s400: and inputting the preprocessed video stream data into a trained offence detection model to realize offence detection. According to the method and the device, the GBCSP module based on the phantom bottleneck detection mechanism is introduced into the model, and the collaborative attention mechanism module is arranged at the output end of the GBCSP module, so that operation violations in video data can be accurately identified.
Description
Technical Field
The invention belongs to the technical field of image detection, and particularly relates to a method and a device for detecting illegal behaviors on a working site, a storage medium and electronic equipment.
Background
In the construction field of large-scale gas production field operation areas, safety is the most important and fundamental requirement of workers. The operation sites are numerous, the distribution is wider, the field environment is quite complex, the operation personnel are numerous, the staff responsible for supervision is limited, the emergency can be rapidly, accurately and comprehensively handled, only the postnatal tracing can be realized, the requirement of the real-time safety monitoring of the operation area of the large-scale gas production site cannot be met, and the generated personal loss and property loss can not be recovered. Within these construction work areas, there are often violations such as falls, calls, smoke, etc., which can present potential safety risks to operators and equipment. Therefore, the illegal behavior detection system based on the deep learning technology gradually becomes an effective solution, and the existing behavior recognition algorithm has the problems that the image feature extraction precision is low due to the fact that the network parameter quantity is too large and the small target information is difficult to recognize.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method for detecting the operation site violations, which can extract small target information in an image, so that the identification accuracy of the operation site violations can be improved.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a method for detecting operation site illegal behaviors comprises the following steps:
s100: collecting video stream data of an operation site;
s200: preprocessing the video stream data;
s300: constructing an illegal behavior detection model and training;
s400: and inputting the preprocessed video stream data into a trained offence detection model to realize offence detection.
Preferably, in step S200, preprocessing the video stream data includes the following steps: the picture is translated, rotated, clipped and gaussian noise added.
Preferably, in step S300, the offence detection model further includes a result output detection head.
Preferably, in step S300, the rule-breaking behavior detection model is trained by the following steps:
s301: collecting video data sets containing illegal behaviors, preprocessing, marking the preprocessed video data sets, and dividing the marked video data sets into training sets and test sets;
s302: setting model training parameters, inputting a training set into a model to train, and finishing model training when the model precision reaches 90% and the model running speed is within 2 seconds in the training process;
s303: testing the trained model by using a test set, evaluating the model by using evaluation indexes including an accuracy rate, a recall rate and an F1 value in the test process, and passing the test when the accuracy rate evaluation value, the recall rate evaluation value and the F1 evaluation value reach 0.9; otherwise, the model parameters are adjusted to train the model again.
The invention also provides a device for detecting the illegal actions on the operation site, which comprises the following steps:
the acquisition module is used for acquiring video stream data of the operation site;
the preprocessing module is used for preprocessing the video stream data;
the model construction and training module is used for constructing an offence detection model and training; the system comprises a fault behavior detection model, a fault behavior detection model and a fault behavior detection model, wherein the fault behavior detection model comprises a backbone network and a feature fusion network, the backbone network introduces a GBCSP module based on a phantom bottleneck detection mechanism, and a collaborative attention mechanism module is arranged at the output end of the GBCSP module; the feature fusion network extracts higher-level and abstract feature representations by adopting a bidirectional path;
the detection module is used for inputting the preprocessed video stream data into a trained illegal action detection model so as to realize illegal action detection.
Preferably, the device further comprises an alarm module for alarming when the detection module detects the illegal action.
The present invention also provides an electronic device including:
a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein,
the processor, when executing the program, implements a method as described in any of the preceding.
The invention also provides a computer storage medium storing computer executable instructions for performing a method as described in any one of the preceding claims.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention is highly suitable for the complex background environment of the large-scale gas production station operation area;
2. the feature extraction module is based on the proposed GBCSP module, introduces a cooperative attention mechanism, has smaller parameter quantity, has algorithm precision of more than 90%, and can more accurately identify smaller targets compared with other algorithms.
Drawings
FIG. 1 is a flow chart of a method for detecting job site violations according to one embodiment of the present invention;
FIG. 2 is a schematic diagram of a model for detecting violations according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a backbone network of an offence detection model according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a phantom bottleneck structure;
fig. 5 is a schematic structural diagram of a GBCSP module according to another embodiment of the invention;
FIG. 6 is a schematic diagram of a cooperative attention mechanism module CA according to another embodiment of the present invention;
fig. 7 is a schematic structural diagram of a feature fusion network DWBiPFN according to another embodiment of the invention;
fig. 8 is a schematic diagram of the structure of EIOU Loss according to another embodiment of the present invention.
Detailed Description
Specific embodiments of the present invention will be described in detail below with reference to fig. 1 to 8. While specific embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
It should be noted that certain terms are used throughout the description and claims to refer to particular components. Those of skill in the art will understand that a person may refer to the same component by different names. The specification and claims do not identify differences in terms of components, but rather differences in terms of the functionality of the components. As used throughout the specification and claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. The description hereinafter sets forth a preferred embodiment for practicing the invention, but is not intended to limit the scope of the invention, as the description proceeds with reference to the general principles of the description. The scope of the invention is defined by the appended claims.
For the purpose of facilitating an understanding of the embodiments of the present invention, reference will now be made to the drawings, by way of example, and specific examples of which are illustrated in the accompanying drawings.
In one embodiment, as shown in fig. 1, the present invention provides a method for detecting a violation on a job site, including the following steps:
s100: collecting video stream data of an operation site;
s200: preprocessing the video stream data;
s300: constructing an illegal behavior detection model and training;
s400: and inputting the preprocessed video stream data into a trained offence detection model to realize offence detection.
In another embodiment, in step S200, preprocessing the video stream data includes the following steps: the picture is translated, rotated, clipped and gaussian noise added.
In another embodiment, as shown in fig. 2, the offence detection model includes: a Backbone network (Backbone), a feature fusion network (negk), and a result output detection head.
In this embodiment, as shown in fig. 3, the backbone network includes, connected in order:
input layers (640,640,3) (three quantities correspond to width, height, number of channels, respectively);
a Focus layer (320,320,12);
Conv2D_BU_SiLU layer (320,320,64);
GBCSP layer (160,160,128);
Conv2D_BU_SiLU layer (80,80,256);
GBCSP layer (80,80,256);
Conv2D_BU_SiLU layer (40,40,512);
GBCSP layer (40,40,512);
Conv2D_BU_SiLU layer (20,20,1024);
sppbottcleck layer (20,20,1024);
GBCSP layer (20,20,1024).
The embodiment creatively proposes that a GBCSP module based on a phantom bottleneck detection mechanism is introduced into a backbone network (the structure of the phantom bottleneck detection mechanism is shown as a figure 4, in the figure 4, the phantom bottleneck detection mechanism consists of two continuous Ghost modules (Ghost modules), a batch normalization BN layer and a ReLU function are connected after the first Ghost module, a batch normalization BN layer is connected after the second Ghost module, the first Ghost module adopts channel separation and expansion operation to process a feature map, the processed feature map is used as the input of a second Ghost module through the batch normalization BN layer and the ReLU function, and the second Ghost module is used for matching and reducing channels through shortcut connection to superimpose (Add) the input and the output, so that information transfer between the two Ghost modules is realized.
Specifically, as shown in fig. 5, the GBCSP module includes two branches, the first branch including a CBS layer (composed of a convolution operation Conv, bulk normalized BN and a SiLU activation function), a Ghost Bottleneck layer and a Conv layer, and the branches performing a phantom Bottleneck stack operation; the second branch comprises a Conv layer, the branch is directly spliced with the result calculated by the first branch through a Concat function without calculation, and then channel integration is carried out through a batch normalization BN layer, a leak Relu function and a CBS layer in sequence. In the GBCSP module, the model can benefit from the characteristics of different levels and different sources at the same time by performing cross-level splicing and channel integration on the two branches, so that the learning capability of the model can be enhanced. The phantom bottleneck structure allows the model to communicate and interact at different levels, enabling the model to better capture features of different scales and levels of abstraction. The multi-level and multi-scale feature representation is beneficial to improving the learning capacity and representation capacity of the model, so that the model obtains better performance in more complex tasks, and further, the image features including color features, texture features, shape features and the like in the image can be better extracted.
In addition, in order to better extract small target information (such as smoking, etc.), the present embodiment further introduces a cooperative attention mechanism module (Coordinate Attention, CA) at the output end of each GBCSP module shown in fig. 3, and compared with the attention mechanism CBAM and the attention mechanism SE, the use of the cooperative attention mechanism module CA according to the present invention not only considers the relationship between channels and spaces, but also considers the importance of pixel location information, so that small target information can be better extracted.
Specifically, as shown in fig. 6, the collaborative attention mechanism module CA includes an input layer (Residual) after Residual connection, so as to input the input characteristics of the number of channels, the height and the width of c×h×w respectively, where the input layer connects a parallel horizontal global average pooling layer and vertical global average pooling layers X Avg Pool and Y Avg Pool, where the channel compression ratio of the global average pooling layer is r. The global average pooling in the horizontal and vertical directions is directly spliced through Concat functions, feature extraction is carried out through Conv2d layers, batch normalization is carried out on BN layers and Non-linear activation layers, and finally weights (Re-weights) of important features are output through parallel modules respectively composed of Conv2d layers and sigmoid functions and are fused with input CxH2xW. The collaborative attention mechanism module CA decomposes the global pooling to take into account the pixel location information in the image, and encodes all channels in both the horizontal (H, 1) and vertical (1, W) directions using two one-dimensional features, where H takes a value from 1 to H and W takes a value from 1 to W. Thus, for each channel C from 1 to C, where C is valued from 1 to C, the output of the C-th channel at height h and width wAnd->Expressed as:
wherein uppercase H and uppercase W denote the size of the pooled core,and->Output of the c-th channel, height h and width w are respectively represented; x is x c (h, i) represents the input at the c-th channel, coordinates (h, i), to the global average pooling layer X Avg Pool; y is c (j, w) represents the input at the c-th channel, coordinate (j, w), to the global average pooling layer Y Avg Pool. The cooperative attention mechanism module CA uses a 1 x 1 convolution transform function F 1 Transforming to generate an intermediate feature mapThe spatial information in the horizontal and vertical directions is encoded, and the calculation formula is as follows:
wherein the handles are along the spatial dimensionIs divided into->(i.e. f h ) And->（f w ) Two tensors, +.>For spatial tandem operation, δ is the activation function. Then through two 1X 1 convolution transformation functionsF h And F w Performing a transformation operation to make f h F of sum w The number of channels is the same as the input, and the result is shown in the following formula:
wherein g h Representing the transform function F according to convolution h For f h The weight mapping, g, is further implemented according to the mapping transformation sigma after the transformation operation w Representing the transform function F according to convolution w For f w The transformation operation is further followed by a weight mapping implemented according to a mapping transformation sigma.
Finally, for g h And g w And expanding to be used as the attention weight. For the c-th channel, the output y of the co-attention mechanism module CA c (i, j) is expressed as:
wherein x is c (i, j) represents the input at coordinates (i, j) at the c-th channel,g represents g h Represents the value of the c-th channel at line i,/for the c-th channel>Representing the value at the jth column for the c-th channel.
In summary, the backbone network introduced into the GBCSP module and the collaborative attention mechanism module CA considers the pixel position information of the image, so that small target information in the image can be better extracted, and the extraction accuracy of the model to the input image can be further enhanced.
Further, the present embodiment proposes a DWBiFPN network (De-Weighted Bidirectional FPN) shown in fig. 7 as a feature fusion network (negk) based on a bi-directional feature pyramid network.
Specifically, the feature fusion network includes three branches, the first branch including a Conv2D (80,80,255) layer and a Concat-Conv2D (80,80,255) layer; the second branch includes Conv2D (40,40,512) layer, concat-Conv2D (40,40,512) layer, and Concat-Conv2D (40,40,512) layer; the third branch includes a Conv2D (20,20,1024) layer and a Concat-Conv2D (20,20,1024) layer.
The feature fusion network treats each bi-directional path (the first bi-directional path includes an up-sampling path and a down-sampling path between the first branch and the second branch; the second bi-directional path includes an up-sampling path and a down-sampling path between the second branch and the third branch) as a feature network layer and repeats the same layer multiple times to enable higher level feature fusion, by repeating the same layer multiple times and feature fusion in each of the repeated layers, the network can utilize and fuse features at different levels to progressively extract higher level and abstract feature representations. The structure can fuse deep semantic features and shallow semantic features to finally generate a feature pyramid with multi-scale information, and can enhance the learning ability of the model, so that the model can better understand and represent input data and obtain better performance in complex tasks. The weight part is removed on the basis of the bidirectional feature pyramid network, so that the calculation amount and the parameter number of the network are reduced.
Further, the result output detection Head adopts coupled output to output three different Head for multi-scale detection, namely 80×80, 40×40 and 20×20, wherein each Head comprises two branches, one branch outputs a human body detection frame, and the other branch outputs an attitude estimation result. The human body detection frame uses EIOULoss as a regression Loss function of the boundary frame (the calculation principle is shown in fig. 8), in fig. 8, EIoU introduces center distance and wide-height Loss, and minimum circumscribed rectangle is replaced by Euclidean distance for minimizing the center point of the target frame, and meanwhile, calculation of the width and the height of the target frame is separated. EIoU loss function L αEIoU Consists of three parts: overlap loss L IoU Center distance loss L dis Loss of width and height L asp The calculation formula is as follows:
wherein c w 、c h Representing the width and height of the smallest bounding rectangle of the prediction frame and the real frame respectively, c represents the diagonal distance of the smallest bounding rectangle of the prediction frame and the real frame, b gt Representing the center points of the predicted and real frames respectively,representing the Euclidean distance, w, of the center point gt Respectively representing the width, h and h of the prediction frame and the real frame gt Representing the heights of the prediction and real boxes, respectively, α represents a given hyper-parameter, and IoU represents the cross-over ratio.
The calculation method has the advantages that not only the overlapping area of the boundary frame regression is considered, but also the distance between the center points of the boundary frames is calculated, and the length-width ratio loss is split into the sum of the length-width loss of the prediction frame and the real frame, so that the model convergence speed is higher.
In another embodiment, in step S300, the offence detection model is trained by:
s301: collecting and preprocessing video data sets containing illegal behaviors (three behaviors of falling, calling and smoking are defined as operation site illegal behaviors) and marking the preprocessed video data sets through a target frame (namely, marking the falling, calling and smoking behaviors in the video data), and dividing the marked video data sets into a training set and a testing set;
s302: setting model training parameters, such as Batch size of the model to 8, learning rate to 0.0005, training the model by using a training set, and finishing model training when model accuracy reaches 90% or more and model running speed is within 2 seconds in the training process;
s303: testing the trained model by using a test set, evaluating the model by using evaluation indexes including an accuracy rate, a recall rate and an F1 value in the test process, and passing the test when the accuracy rate evaluation value, the recall rate evaluation value and the F1 evaluation value reach 0.9; otherwise, the model parameters are adjusted to train the model again, for example, the Batch size of the model is adjusted from 8 to 16, the learning rate is adjusted from 0.0005 to 0.00005, and an optimizer used by the model is adjusted until the model test passes.
In another embodiment, the present invention further provides a device for detecting a violation on a job site, including:
the acquisition module is used for acquiring video stream data of the operation site;
the preprocessing module is used for preprocessing the video stream data;
the model construction and training module is used for constructing an offence detection model and training; the system comprises a fault behavior detection model, a fault behavior detection model and a fault behavior detection model, wherein the fault behavior detection model comprises a backbone network and a feature fusion network, the backbone network introduces a GBCSP module based on a phantom bottleneck detection mechanism, and a collaborative attention mechanism module is arranged at the output end of the GBCSP module; the feature fusion network extracts higher-level and abstract feature representations by adopting a bidirectional path;
the detection module is used for inputting the preprocessed video stream data into a trained illegal action detection model so as to realize illegal action detection.
In another embodiment, the device further comprises an alarm module for alarming when the detection module detects the illegal action.
In another embodiment, the present invention further provides an electronic device, including:
a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein,
the processor, when executing the program, implements a method as described in any of the preceding.
In another embodiment, the present invention also provides a computer storage medium storing computer-executable instructions for performing a method as described in any one of the preceding claims.
It will be appreciated by persons skilled in the art that the above embodiments are provided for clarity of illustration only and are not intended to limit the scope of the invention. Other variations or modifications will be apparent to persons skilled in the art from the foregoing disclosure, and such variations or modifications are intended to be within the scope of the present invention.
Claims (5)
1. A method for detecting operation site violations, the method comprising the steps of:
s100: collecting video stream data of an operation site;
s200: preprocessing the video stream data;
preprocessing the video stream data comprises the following steps: translating, rotating, shearing and adding Gaussian noise to the picture;
s300: constructing an illegal behavior detection model and training;
the system comprises a fault behavior detection model, a fault behavior detection model and a fault behavior detection model, wherein the fault behavior detection model comprises a backbone network and a feature fusion network, the backbone network introduces a GBCSP module based on a phantom bottleneck detection mechanism, and a collaborative attention mechanism module is arranged at the output end of the GBCSP module; the feature fusion network extracts higher-level and abstract feature representations by adopting a bidirectional path;
specifically, the backbone network includes sequentially connected:
an Inputs layer 640,640,3;
focus layer 320,320,12;
Conv2D_BU_SiLU layer 320,320,64;
GBCSP layer 160,160,128;
Conv2D_BU_SiLU layer 80,80,256;
GBCSP layer 80,80,256;
Conv2D_BU_SiLU layer 40,40,512;
GBCSP layer 40,40,512;
Conv2D_BU_SiLU layer 20,20,1024;
sppbottcleck layer 20,20,1024;
GBCSP layer 20,20,1024;
the GBCSP layer comprises two branches, wherein the first branch comprises a CBS layer, a Ghost Bottleneck layer and a Conv layer, the CBS layer is composed of a convolution operation Conv, batch normalization BN and SiLU activation functions, and the branches perform phantom Bottleneck stacking operation; the second branch comprises a Conv layer, the branch is directly spliced with the result calculated by the first branch through a Concat function without calculation, and then channel integration is carried out through a batch normalization BN layer, a leak Relu function and a CBS layer in sequence;
the output end of the GBCSP layer is introduced with a cooperative attention mechanism module, the cooperative attention mechanism module comprises an input layer after residual connection, input characteristics of the number of input channels, the height and the width of the input layer are respectively C multiplied by H multiplied by W, the input layer is connected with a parallel horizontal global average pooling layer X Avg Pool and a vertical global average pooling layer Y Avg Pool, the channel compression ratio of the global average pooling layer is r, the horizontal and vertical global average pooling is directly spliced through a Concat function, the characteristics are extracted through a Conv2d layer, the weights of important characteristics are output through a module composed of a Conv2d layer and a Non-linear activation layer respectively, and the input is fused with the input C multiplied by H multiplied by W;
the feature fusion network comprises three branches, wherein the first branch comprises a Conv2D layer and a Concat-Conv2D layer; the second branch comprises a Conv2D layer, a Concat-Conv2D layer and a Concat-Conv2D layer; the third branch comprises Conv2D layer and Concat-Conv2D layer;
the feature fusion network regards a first bidirectional path formed by an up-sampling path and a down-sampling path between a first branch and a second bidirectional path formed by an up-sampling path and a down-sampling path between a second branch and a third branch as a feature network layer, and repeats the same layer a plurality of times to enable higher-level feature fusion, and by repeating the same layer a plurality of times and performing feature fusion in each repeated layer, the network can utilize and fuse features on different levels to gradually extract higher-level and abstract feature representations;
the illegal behavior detection model further comprises an output detection Head, wherein the output detection Head adopts coupled output to output three different heads for multi-scale detection, namely 80×80, 40×40 and 20×20, each Head comprises two branches, one branch outputs a human body detection frame, and the other branch outputs a gesture estimation result;
the offence detection model is trained by the following steps:
s301: collecting video data sets containing illegal behaviors, preprocessing, marking the preprocessed video data sets, and dividing the marked video data sets into training sets and test sets;
s302: setting model training parameters, inputting a training set into a model to train, and finishing model training when the model precision reaches 90% and the model running speed is within 2 seconds in the training process;
s303: testing the trained model by using a test set, evaluating the model by using evaluation indexes including an accuracy rate, a recall rate and an F1 value in the test process, and passing the test when the accuracy rate evaluation value, the recall rate evaluation value and the F1 evaluation value reach 0.9; otherwise, adjusting the model parameters to train the model again;
s400: and inputting the preprocessed video stream data into a trained offence detection model to realize offence detection.
2. A job site violation detection device for implementing the method of claim 1, the device comprising:
the acquisition module is used for acquiring video stream data of the operation site;
the preprocessing module is used for preprocessing the video stream data;
the model construction and training module is used for constructing an offence detection model and training; the system comprises a fault behavior detection model, a fault behavior detection model and a fault behavior detection model, wherein the fault behavior detection model comprises a backbone network and a feature fusion network, the backbone network introduces a GBCSP module based on a phantom bottleneck detection mechanism, and a collaborative attention mechanism module is arranged at the output end of the GBCSP module; the feature fusion network extracts higher-level and abstract feature representations by adopting a bidirectional path;
the detection module is used for inputting the preprocessed video stream data into a trained illegal action detection model so as to realize illegal action detection.
3. The apparatus of claim 2, further comprising an alarm module for alerting when the detection module detects an violation.
4. An electronic device, comprising:
a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein,
the processor, when executing the program, implements the method of claim 1.
5. A computer storage medium storing computer executable instructions for performing the method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311490614.8A CN117237993B (en) | 2023-11-10 | 2023-11-10 | Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311490614.8A CN117237993B (en) | 2023-11-10 | 2023-11-10 | Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117237993A CN117237993A (en) | 2023-12-15 |
| CN117237993B true CN117237993B (en) | 2024-01-26 |
Family
ID=89098525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311490614.8A Active CN117237993B (en) | 2023-11-10 | 2023-11-10 | Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237993B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022037280A1 (en) * | 2020-08-19 | 2022-02-24 | 广西电网有限责任公司贺州供电局 | Multi-channel cnn based method for detecting power transformation field operation violations |
| CN114782892A (en) * | 2022-04-20 | 2022-07-22 | 上海东普信息科技有限公司 | Illegal behavior target detection method, device, equipment and storage medium |
| CN114913606A (en) * | 2022-06-17 | 2022-08-16 | 中国计量大学 | YOLO-based violation detection method for deep learning industrial field production work area |
| CN115015911A (en) * | 2022-08-03 | 2022-09-06 | 深圳安德空间技术有限公司 | Method and system for manufacturing and using navigation map based on radar image |
| CN115471721A (en) * | 2022-09-07 | 2022-12-13 | 深圳大学 | Image target detection method, system, electronic device and storage medium |
| CN115830392A (en) * | 2022-12-20 | 2023-03-21 | 北方民族大学 | Student behavior identification method based on improved YOLOv5 |
| CN116863534A (en) * | 2023-05-29 | 2023-10-10 | 贵州电网有限责任公司 | High-precision illegal behavior detection method and device for transformer substation scene |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0905317D0 (en) * | 2008-07-14 | 2009-05-13 | Musion Ip Ltd | Video processing and telepresence system and method |
-
2023
- 2023-11-10 CN CN202311490614.8A patent/CN117237993B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022037280A1 (en) * | 2020-08-19 | 2022-02-24 | 广西电网有限责任公司贺州供电局 | Multi-channel cnn based method for detecting power transformation field operation violations |
| CN114782892A (en) * | 2022-04-20 | 2022-07-22 | 上海东普信息科技有限公司 | Illegal behavior target detection method, device, equipment and storage medium |
| CN114913606A (en) * | 2022-06-17 | 2022-08-16 | 中国计量大学 | YOLO-based violation detection method for deep learning industrial field production work area |
| CN115015911A (en) * | 2022-08-03 | 2022-09-06 | 深圳安德空间技术有限公司 | Method and system for manufacturing and using navigation map based on radar image |
| CN115471721A (en) * | 2022-09-07 | 2022-12-13 | 深圳大学 | Image target detection method, system, electronic device and storage medium |
| CN115830392A (en) * | 2022-12-20 | 2023-03-21 | 北方民族大学 | Student behavior identification method based on improved YOLOv5 |
| CN116863534A (en) * | 2023-05-29 | 2023-10-10 | 贵州电网有限责任公司 | High-precision illegal behavior detection method and device for transformer substation scene |
Non-Patent Citations (6)
| Title |
|---|
| Foxtail Millet Ear Detection Method Based on Attention Mechanism and Improved YOLOv5;Shujin Qiu 等;《sensors》;1-14 * |
| Ghost shuffl lightweight pose network with effective feature representation and learning for human pose estimation;Senquan Yang 等;《IET Computer Vision》;525-540 * |
| GhostNet: More Features from Cheap Operations;Kai Han 等;《arXiv》;1-10 * |
| 基于深度学习的苹果花识别检测方法研究;陈国防;《中国优秀硕士学位论文全文数据库 农业科技辑》(第1期);第D048-352页，正文第4.4.1节 * |
| 基于深度神经网络的分布式视频违规行为检测系统实现;李柯江;《中国优秀硕士学位论文全文数据库 信息科技辑》(第7期);I136-675 * |
| 采用注意力机制与改进YOLOv5的光伏用地检测;陈笛 等;《自然资源遥感》;第1-6页，正文第1.2节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117237993A (en) | 2023-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110728209B (en) | Gesture recognition method and device, electronic equipment and storage medium | |
| CN109657592B (en) | Face recognition method of intelligent excavator | |
| CN109948453B (en) | Multi-person attitude estimation method based on convolutional neural network | |
| CN109858367B (en) | Visual automatic detection method and system for worker through supporting unsafe behaviors | |
| CN110263603B (en) | Face recognition method and device based on central loss and residual error visual simulation network | |
| CN109598211A (en) | A kind of real-time dynamic human face recognition methods and system | |
| CN112560745B (en) | Method for discriminating personnel on electric power operation site and related device | |
| CN116152863B (en) | Personnel information identification method and device, electronic equipment and storage medium | |
| CN111062373A (en) | Hoisting process danger identification method and system based on deep learning | |
| CN114998830A (en) | Wearing detection method and system for safety helmet of transformer substation personnel | |
| CN111738074B (en) | Pedestrian attribute identification method, system and device based on weak supervision learning | |
| CN114565891A (en) | Smoke and fire monitoring method and system based on graph generation technology | |
| CN114170686A (en) | Elbow bending behavior detection method based on human body key points | |
| CN117237993B (en) | Method and device for detecting operation site illegal behaviors, storage medium and electronic equipment | |
| CN117152844A (en) | High-integrity worker construction attitude detection method and system based on computer vision | |
| CN115187660A (en) | Knowledge distillation-based multi-person human body posture estimation method and system | |
| CN114399729A (en) | Monitoring object movement identification method, system, terminal and storage medium | |
| Gong et al. | Human elbow flexion behaviour recognition based on posture estimation in complex scenes | |
| CN115937506B (en) | Method, system, equipment and medium for locating bridge side falling point hole site information | |
| CN117496131B (en) | Electric power operation site safety behavior identification method and system | |
| LU501401B1 (en) | Action risk identification method for power system field operation based on graph convolution | |
| Lin et al. | An integrated application platform for safety management and control of smart power plants based on video image technology | |
| CN112712126B (en) | Picture identification method | |
| Rahman et al. | Outdoor social distancing violation system detection using YOLO algorithm | |
| CN115359537A (en) | Face key point detection method, face key point detection device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |