CN114898181A - Method and device for identifying hidden dangers and violations of regulations for explosive-related videos - Google Patents

Abstract

The invention discloses a hidden danger violation identification method and a hidden danger violation identification device for an explosion-related video, which relate to the field of explosion construction, and the method comprises the steps of S1 constructing a video authenticity detection model, a video target identification model and an identity identification model; s2, constructing an annotation database and a face information database; s3, training and optimizing a video target recognition model and an identity recognition model; s4, acquiring the explosion-related video; s5, carrying out authenticity analysis on the explosion-related video, and identifying related information of the blasting operation and information of blasting operators; s6, analyzing hidden dangers and violation conditions in the blasting video; the video true and false detection model, the optimized video target identification model and the optimized identity identification model are used for analyzing the explosion-related video, analyzing and judging the true and false of the explosion-related video, the related operation information in the explosion process and the information of operators operating the explosion, judging whether potential safety hazards exist in the process operation or not according to the analysis results, and standardizing the operation of the operators.

Description

Method and device for identifying hidden dangers and violations of regulations for explosive-related videos

technical field

The invention relates to the field of blasting construction, in particular to a method and device for identifying hidden dangers and violations of regulations for blasting-related videos.

Background technique

In the field of safety supervision of petroleum exploration technology, how to ensure the work quality, process standardization and personal safety of blasters and constructors during civil blasting operations is a technical problem in this field.

Most of the existing technologies are for construction personnel to bring information collection terminal equipment, carry out standardized operations and completely record the operation process, submit the operation video data to the information collection center on the same day, and manually judge whether the construction process is qualified, such as: not wearing a helmet, In the process of judging, the staff needs to review a large number of videos every day, and visual fatigue will cause missed inspections and missed reports. Human judgment is subjective, and the auditing of data cannot strictly conform to the established standards. At the same time, the traditional technology wastes a lot of human, material and financial resources.

SUMMARY OF THE INVENTION

The purpose of the present invention is to design a method and device for identifying hidden dangers and violations of regulations for explosive-related videos in order to solve the above problems.

The present invention realizes above-mentioned purpose through following technical scheme:

Hidden and illegal identification methods for explosive-related videos, including:

S1. Build a video authenticity detection model, a video target recognition model and an identity recognition model. The video target recognition model is a YOLOv5 detection model, and the identity recognition model is a face recognition model;

S2. Build a labeling database and a face information database, label the training data set and export the labeling file to form a labeling database, and the face information database stores relevant information about the licensed civilian explosive operators;

S3. Use the annotation database to train and optimize the video target recognition model, and use the face information database to train and optimize the identity recognition model;

S4. Obtain the explosion-related video to be analyzed;

S5. Import the explosion-related video to be analyzed into the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model, and perform authenticity analysis on the explosion-related video, identify the relevant information of the blasting operation, and identify the blasting operation. information about personnel;

S6. Analyze the authenticity of the blasting-related video, identify the relevant information of blasting operations and identify the information of blasting operators, and analyze the hidden dangers and violations of regulations in the blasting video.

The device for identifying hidden dangers and violations of laws and regulations used for explosive-related videos, including the bottom-level server, the bottom-level server includes:

storage; storage for storing computer programs;

A processor; the processor is used to execute a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned method for identifying hidden dangers and violations of regulations for explosive-related videos.

The beneficial effect of the present invention is that: the video authenticity detection model, the optimized video target recognition model and the optimized identity recognition model are used to analyze the explosion-related video, so as to analyze and judge the authenticity of the explosion-related video and the quality of the explosion-related video. Relevant operation information and information of operators who operate blasting, and through these analysis results to determine whether there are safety hazards in the process operation, standardize the operation of the staff, realize efficient operation, accurate judgment, intelligent analysis, and make the management and control of safety issues more efficient. , saving a lot of human, material and financial resources.

Description of drawings

1 is a schematic diagram of a video target recognition model in the present invention;

Fig. 2 is the flow chart of analyzing the cleaning degree of video related to explosion in the present invention;

Fig. 3 is the identification flow chart of the identity identification model in the present invention;

Fig. 4 is the detection flow chart of the video authenticity detection model in the present invention;

Fig. 5 is the RNN character recognition framework of the present invention;

FIG. 6 is a single-layer bidirectional RNN network of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, embodiments of the present invention. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In the description of the present invention, it should be understood that the orientations or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "left", "right", etc. are based on those shown in the accompanying drawings The orientation or positional relationship, or the orientation or positional relationship that the product of the invention is usually placed in use, or the orientation or positional relationship that is commonly understood by those skilled in the art, are only for the convenience of describing the present invention and simplifying the description, rather than indicating or It is implied that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and therefore should not be construed as a limitation of the invention.

Furthermore, the terms "first", "second", etc. are only used to differentiate the description and should not be construed to indicate or imply relative importance.

In the description of the present invention, it should also be noted that, unless otherwise expressly specified and limited, terms such as "arrangement" and "connection" should be understood in a broad sense, for example, "connection" may be a fixed connection or Detachable connection, or integral connection; may be mechanical connection or electrical connection; may be direct connection or indirect connection through an intermediate medium, and may be internal communication between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Hidden and illegal identification methods for explosive-related videos, including:

S1. Build a video authenticity detection model, a video target recognition model, and an identity recognition model. The video target recognition model is a YOLOv5 detection model, and the identity recognition model is a face recognition model.

S2. Build a labeling database and a face information database, label the training data set and export the labeling file to form a labeling database, and the face information database stores relevant information of licensed civilian explosive operators.

S3. Use the annotation database to train and optimize the video target recognition model, and use the face information database to train and optimize the identity recognition model.

S4. Obtain the explosion-related video to be analyzed.

S0, analyze and evaluate the clarity of the images in the explosion-related video to be analyzed, and filter them out; as shown in Figure 2, specifically including:

N为图像序列长度，f(i,j)代表一幅大小为N×N的矩阵，其中i＝0,1,2,···,N-1和j＝0,1,2,···,N-1，F(k,l)表示f(i,j)的傅里叶变换S01. Perform Fourier transform of the image to be analyzed in the explosion-related video into the frequency domain,

N is the length of the image sequence, f(i,j) represents a matrix of size N×N, where i=0,1,2,...,N-1 and j=0,1,2,... ·,N-1, F(k,l) represents the Fourier transform of f(i,j)

S02, remove the low frequency signal lower than the preset frequency;

S03. Convert the image from the frequency domain to the spatial domain by using the fast Fourier transform,

上述式子中，f(a,b)代表一幅大小为N×N的矩阵，其中a＝0,1,2,···,N-1和b＝0,1,2,···,N-1，f(a,b)表示F(k,l)的逆傅里叶变换。

In the above formula, f(a,b) represents a matrix of size N×N, where a=0,1,2,...,N-1 and b=0,1,2,... , N-1, f(a,b) represents the inverse Fourier transform of F(k,l).

其中，F(k,l)为频域像素均值，P(k,b)为频域形式，N为序列长度，b＝0,1,2,···,N-1。

Among them, F(k,l) is the pixel mean value in the frequency domain, P(k,b) is the frequency domain form, N is the sequence length, b=0,1,2,...,N-1.

S04: Calculate the average amplitude value in the spatial domain, and the graph whose average amplitude value is greater than the preset threshold is a clear image, otherwise it is a blurred image, and the blurred image is eliminated.

S5. Import the explosion-related video to be analyzed into the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model, and perform authenticity analysis on the explosion-related video, identify the relevant information of the blasting operation, and identify the blasting operation. personnel information.

S6. Analyze the authenticity of the blasting-related video, identify the relevant information of blasting operations and identify the information of blasting operators, and analyze the hidden dangers and violations of regulations in the blasting video.

Through the video authenticity detection model, the optimized video target recognition model and the optimized identity recognition model, the explosion-related video is analyzed to analyze and judge the authenticity of the explosion-related video, the relevant operation information in the blasting process, and the operation of the blasting. Operator information, and through these analysis results to determine whether there are safety hazards in the process operation, standardize the operation of the staff, realize efficient operation, accurate judgment, intelligent analysis, make the management and control of safety problems more efficient, and save a lot of manpower and material resources and financial resources;

The conditions for recording videos in the wild are complicated, resulting in uneven video quality. Therefore, before analyzing the video related to the explosion, first analyze and evaluate the clarity of the video image related to the explosion, and quickly filter out the blurred video from a large number of videos, so as to improve the re-inspection. Efficiency; blur reduces the clarity of the image, seriously affects the image quality, and causes difficulties or even failures in image analysis, processing, and reception. Therefore, an effective blur evaluation method must be used to control the use of blurred images, thereby improving the overall performance of the system.

The device for identifying hidden dangers and violations of laws and regulations used for explosive-related videos, including the bottom-level server, the bottom-level server includes:

storage; storage for storing computer programs;

A processor; the processor is used to execute a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned method for identifying hidden dangers and violations of regulations for explosive-related videos.

The hidden danger and violation identification device also includes a collection device, which is used for collecting blasting videos and relevant information of blasting personnel, and the collection device is connected to the underlying server for communication.

The hidden danger and violation identification device further includes a cloud server and a remote terminal, and the signal terminal of the cloud server is respectively connected with the signal terminal of the remote terminal and the signal terminal of the underlying server.

The video target recognition model is the YOLOv5 detection model. As shown in Figure 1, the target detection adopts the first-order target detection algorithm-YOLOv5. This method has a good use of the Pytorch framework and can easily train its own data set. The Pytorch framework is easier to invest Production. Not only is the environment easy to configure, model training is also very fast, and batch inference produces real-time results. Ability to perform efficient inference directly on single images, batch images, videos, and even webcam port inputs. Finally, the detection speed of YOLOv5s is very fast, and the judgment results of a large number of homework videos can be obtained in a short time.

YOLOv5 is a single-stage target detection algorithm. The algorithm adds some new improvement ideas, so that its speed and accuracy have been greatly improved. The main improvement ideas are as follows:

Input: In the model training stage, some improvement ideas are proposed, mainly including Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling;

Benchmark network: Integrate some new ideas in other detection algorithms, mainly including: Focus structure and CSP structure;

Neck network: The target detection network often inserts some layers between the BackBone (backbone network) and the final Head output layer. The FPN+PAN structure is added to YOLOv5;

Head output layer: The anchor box mechanism of the output layer is the same as that of YOLOv4. The main improvements are the loss function GIOU_Loss during training, and the DIOU_nms for prediction box screening.

The identity recognition model is a face recognition model, and the process of face recognition is shown in Figure 3. Face detection: use a face detection model (target detection model or other detection model) to find the position of all faces in the image (the result is detection frame), and cut out the image of the face part;

Face alignment: Use the face alignment model to correct the face, perform correction based on the key points of the face, and align the key points of the standard face;

Face encoding: use deep learning models to encode face images and extract face features;

Identity recognition: Compare the facial features with the data in the face database to determine the identity of the face.

FaceNet is a general-purpose face recognition system: it uses a deep convolutional neural network (CNN) to learn to map images into Euclidean space. Spatial distance is directly related to image similarity: different images of the same person have a small distance in space, and images of different people have a large distance in space, which can be used for face verification, recognition and clustering.

FaceNet uses a deep neural network to extract features, and uses triplet_loss to measure the distance error between samples during training. Before training or during online learning, it constantly creates "difficulties" for the neural network, that is, it is always looking for the "self" that is the least like the sample, and at the same time looking for the "other" that is most like oneself. Through the stochastic gradient descent method, the gap between all of its own samples is continuously shortened, and the gap with others is widened as much as possible, and finally an optimum is achieved. Through such an embedding learning, the original feature extraction network output can be obtained. The layers are further learned to improve the representation of the features.

For the entire FaceNet structure, an extremely deep network, Inception ResNet-v2, is used for feature extraction. The model structure consists of 3 Inception modules with residual connections and 1 Inception v4 module.

The overall framework of the model is basically consistent with other classical deep learning methods. The previous feature extraction part is also based on CNN, but the deep network Inception-v4, followed by a feature normalization layer, makes the feature two-paradigm

That is, the image features are mapped to a hypersphere, which can avoid the differences caused by the imaging environment of the sample. Finally, triplet_loss is used as the loss, and stochastic gradient descent (SDG, Stochastic GradientDescent) is used for backpropagation. The model Inception also connects the residual, which is also one of the prominent points of this method, which improves the training convergence speed.

The video authenticity detection model is an OCR character recognition model, as shown in Figure 4, the video authenticity detection model in the present invention is mainly divided into two parts:

Video authenticity determination: OCR text recognition technology is used to identify whether the watermark time in the lower right corner of the video shooting area is consistent with the operation time of the day, so as to prevent the appearance of spliced videos.

Night construction identification: Use the video shot on the day of the operation to extract the watermark date on the video, determine whether the staff is working at night, prevent this phenomenon, and strengthen safety control.

The video authenticity detection model is the main framework based on the RNN text recognition algorithm, as shown in Figure 5.

(1) The window size in Max pooling is 1*2, which ensures that the proposed feature has a horizontal length, which is conducive to recognizing longer texts;

(2) The training of CNN+RNN is difficult, so BatchNorm is added to help the model converge;

Advantage:

(1) End-to-end training is possible;

(2) There is no need to perform character segmentation and horizontal scaling operations, only need to scale to a fixed length in the vertical direction, and can identify sequences of any length;

(3) Dictionary-based models and arbitrary models that are not dictionary-based can be trained;

(4) The training speed is fast, and the model is small.

The entire CRNN network can be divided into three parts: Convolutional Layers-----The convolutional layer here is an ordinary CNN network, which is used to extract the Convolutional feature maps of the input image, that is, convert the image into a convolutional feature matrix;

Recurrent Layers-----The recurrent network layer here is a deep bidirectional LSTM network, which continues to extract text sequence features on the basis of convolutional features. The so-called deep RNN network refers to an RNN network with more than two layers. For single-layer bidirectional RNN network, the structure is shown in Figure 6,

A second stack-shaped deep bidirectional structure is used in CRNN.

Transcription Layers-----After the RNN output is softmax, it is output for characters.

The invention solves the safety management and control problems in the geophysical exploration process, such as: blasting personnel do not hold a certificate for work, do not wear safety helmets and anti-static clothing, and spray and blast at night. Through model detection, it is judged whether there is a potential safety hazard in the process operation, and the operation of the staff is standardized;

Through the integration of the underlying algorithm and the front-end, efficient operation, accurate judgment, and intelligent analysis are realized, which makes the management and control of security issues more efficient and saves a lot of manpower, material and financial resources.

It integrates target detection technology, OCR text recognition technology, face recognition technology, video fuzzy judgment technology, distributed information processing technology, etc., can judge the standardization of explosive video operations, process the video through the underlying algorithm, and use the front-end function to process the video. The analysis results generated by the platform are saved to the background and sent to the regulatory department or collection center.

The technical solutions of the present invention are not limited to the limitations of the above-mentioned specific embodiments, and all technical modifications made according to the technical solutions of the present invention fall within the protection scope of the present invention.

Claims (7)

1. A hidden danger violation identification method for an implosion video is characterized by comprising the following steps:

s1, constructing a video authenticity detection model, a video target identification model and an identity identification model, wherein the video target identification model is a YOLOv5 detection model, and the identity identification model is a face identification model;

s2, constructing a label database and a face information database, labeling the training data set to export a label file to form a label database, wherein the face information database stores the relevant information of the certified civil explosion operating personnel;

s3, training and optimizing the video target recognition model by adopting the annotation database, and training and optimizing the identity recognition model by adopting the face information database;

s4, acquiring the explosion-related video to be analyzed;

s5, importing the explosion-related video to be analyzed into a video authenticity detection model, an optimized video target identification model and an optimized identity identification model, and carrying out authenticity analysis on the explosion-related video, identification of relevant information of explosion operation and identification of information of explosion operators;

and S6, carrying out authenticity analysis according to the blasting video, identifying relevant information of blasting operation and identifying information of blasting operators, and analyzing hidden dangers and violation conditions in the blasting video.

2. The method for identifying the hidden danger violation for the explosion-related video as recited in claim 1, wherein a step S0 is further included between S4 and S5, and the definition of the image in the explosion-related video to be analyzed is analyzed, evaluated and screened out.

3. The method for identifying a hidden danger violation for an implosion-related video as recited in claim 1, wherein S0 comprises:

s01, carrying out Fourier transformation on the image of the explosion-related video to be analyzed, and converting the image into a frequency domain;

s02, removing low-frequency signals with the frequency lower than the preset frequency;

s03, converting the image from a frequency domain to a space domain by using fast Fourier transform;

and S04, calculating an amplitude mean value in the spatial domain, judging whether the image is a blurred image according to the amplitude mean value, and removing the blurred image.

4. The method for identifying the hidden danger violation for the implosion-related video as recited in claim 1, wherein in S04, the graph with the amplitude mean value larger than the preset threshold is a clear image, otherwise, the graph is a blurred image.

5. A hidden danger identification device violating regulations for concerning with exploding video, its characterized in that includes bottom server, and bottom server includes:

a reservoir; the storage is used for storing a computer program;

a processor; the processor is adapted to execute a computer program, which when executed performs the steps of the method for identification of a hidden violation for an implosion video according to any of claims 1-4.

6. The hidden danger violation identification device for the implosion video as recited in claim 5, further comprising a collection device for collecting the information related to the blasting video and the blasting personnel, wherein the collection device is in communication connection with the underlying server.

7. The hidden danger violation identification device for the implosion-related video according to claim 5, further comprising a cloud server and a remote terminal, wherein a signal end of the cloud server is connected with a signal end of the remote terminal and a signal end of the bottom server respectively.

Images