CN111339927B - An intelligent identification system for the working status of the personnel in the electric power business hall - Google Patents

Abstract

An intelligent recognition system for the working state of personnel in an electric power business hall. Relates to the field of power supply service, in particular to an intelligent recognition system for working states of power business hall personnel. The intelligent recognition system for the working state of the personnel in the electric power business hall is convenient to recognize and reliably judges the working state of the personnel. The system comprises an acquisition module, a training module, a preprocessing module, a target detection module, a logic judgment module and an output alarm module, wherein the acquisition module is used for marking a video in an acquisition business hall and intercepting pictures containing effective information from the video; the training module is used for enhancing the acquired picture data and training the depth network by the enhanced picture data. In work, the invention detects that the working state of the staff in the business hall is wrong in an image recognition mode, and sends abnormal information to remind a manager to prompt the staff to correct in time. The automatic judgment of whether the staff is on duty and whether the work cards are correctly placed in the working hours of the electric power business hall can be realized, and convenience and reliability are realized.

Description

An intelligent identification system for the working status of the personnel in the electric power business hall

technical field

The invention relates to the field of power supply services, in particular to an intelligent identification system for the working status of personnel in electric power business halls.

Background technique

With the development of the economy, the demand for power loads has increased sharply, and the volume of related business transactions has increased significantly. The power supply business hall is the service window of the State Grid Corporation for the public. The scale of the video surveillance system in this scenario is increasing day by day, which exposes the increasingly serious problems of the power supply service quality of the business hall, such as whether the employees are on duty and whether the badges are placed correctly. wait. At present, corresponding work service norms have been formulated, but due to the lack of effective supervision means, it is difficult to effectively supervise the working status of the staff.

At present, object detection is one of the basic tasks in the field of computer vision, and the academic circle has a research history of nearly two decades. In recent years, with the rapid development of deep learning technology, the target detection algorithm has also shifted from the traditional algorithm based on manual features to the detection technology based on deep neural network.

Existing technology: R-CNN and other target detection algorithms first find the candidate frame from the original image, then perform feature extraction on the picture in the candidate frame, and finally use SVM to classify the features. In this way, the accuracy of the result depends on the selection of candidate boxes; and convolution calculations are performed for each candidate box, which reduces the computational efficiency of the model.

Contents of the invention

Aiming at the above problems, the present invention provides an intelligent identification system for the working status of personnel in electric power business halls, which is convenient for identification and reliable for judging the working status of personnel.

The technical solution of the present invention is: comprising acquisition module, training module, preprocessing module, target detection module, logical judgment module and output alarm module,

The collection module is used to collect the video in the business hall, and intercept pictures containing valid information therefrom to mark;

The training module is used to enhance the collected image data and train the deep network with the enhanced image data;

The preprocessing module is used to perform image preprocessing on the received picture;

The target detection module is used to perform target detection on images;

The logic judgment module is used to judge the working status of the station and the personnel in the station through the target found in the image;

The output alarm module is used to convert the logical judgment result and provide early warning.

The targets in the target detection module include displays, seats, staff and badges.

Described training module comprises the following steps:

1) Selection and labeling of sample sets;

2) Build a Faster RCNN network structure;

3) Training Faster RCNN;

4) Detect Faster RCNN.

The selection label of the sample set in step 1):

Use the graphic labeling tool LabelImg to label the sample set, and manually circle each sample through LabelImg to obtain the XML labeling file of each sample data corresponding to the picture; randomly select the training set, verification set and test set from all the data in proportion .

Build the Faster RCNN network structure in step 2):

The network structure of Faster RCNN is constructed in the deep learning system TensorFlow. The network structure includes: input layer, pooling layer, convolution layer and output layer.

Steps to train Faster RCNN in step 3):

Step 3.1), on the already trained model, train the RPN network, corresponding to stage1_rpn_train.pt, use the trained RPN network to collect proposals, corresponding to rpn_test.pt;

Step 3.2), train the Fast RCNN network for the first time, corresponding to stage1_fast_rcnn_train.pt;

Step 3.3), the second training RPN network, corresponding to stage2_rpn_train.pt, again using the trained RPN network, collecting proposals, corresponding to rpn_test.pt;

Step 3.4), train the Fast RCNN network for the second time, corresponding to stage2_fast_rcnn_train.pt.

Detect Faster RCNN in step 4):

First, use the Resnet/VGG network structure for basic feature extraction;

Secondly, the RPN network is responsible for calculating the coordinates of the area where the target may exist and judging whether it is the foreground/background and the target area obtained by using the RPN network and then passing through the ROI Pooling layer to obtain a feature vector of the same length;

Finally, the softmax classifier is connected through two fully connected layers, in which the fully connected layer realizes specific classification and more accurate regression coordinates.

When the present invention detects that the working state of the personnel in the business hall is wrong through the image recognition mode during work, it sends abnormal information to remind the management personnel to urge the employees to make corrections in time. It can automatically judge whether the employees are on duty during the working hours of the power business hall and whether the badges are placed correctly, which is convenient and reliable.

Description of drawings

Fig. 1 is a block diagram of the present invention,

Fig. 2 is the training schematic diagram of Faster RCNN in the present invention,

Fig. 3 is the detection schematic diagram of Faster RCNN in the present invention,

Fig. 4 is a logic judgment diagram of the present invention.

Detailed ways

As shown in Figures 1-4, the present invention includes an acquisition module, a training module, a preprocessing module, a target detection module, a logic judgment module and an output alarm module,

The collection module is used to collect the video in the business hall, and intercept pictures containing valid information therefrom to mark;

The training module is used to enhance the collected image data and train the deep network with the enhanced image data;

First, a large number of videos are collected in all electric power business halls, and pictures containing effective information are intercepted from them for labeling. Data enhancement is performed through this module and a deep network is trained with the enhanced picture data. In this way, the training of deep neural network can be realized.

The preprocessing module is used to perform image preprocessing on the received picture;

In this way, the image preprocessing is mainly performed on the received image, and the image can be processed more easily by changing the size of the image, normalizing and other preprocessing methods.

The target detection module is used to perform target detection on images;

By training the generated model, find objects of interest in the input image, such as people and badges.

The logic judgment module is used to judge the working status of the station and the personnel in the station through the target found in the image;

In this way, after the target detection result is obtained, the customer information is excluded, the workstation in the image is generated, and the working status of the staff in the workstation is judged.

The output alarm module is used to convert the logical judgment result and provide early warning; that is, the logical judgment result is converted into alarm information and returned to the front end through the http interface.

The targets in the target detection module include displays, seats, staff and badges. It is convenient to select different targets and improve the reliability of recognition.

Described training module comprises the following steps:

1) Selection and labeling of sample sets;

2) Build a Faster RCNN network structure;

3) Training Faster RCNN;

4) Detect Faster RCNN.

The selection label of the sample set in step 1):

Use the graphic labeling tool LabelImg to label the sample set, and manually circle each sample through LabelImg to obtain the XML labeling file of the picture corresponding to each sample data; randomly select the training set, verification set and test set from all the data in proportion . Among them, the training set is used to estimate the model, the verification set is used to determine the network structure or the parameters that control the complexity of the model, and the test set is used to test the performance of the final selection of the optimal model. Experiments show that the detection effect is the best when the ratio is 85:10:5.

Build the Faster RCNN network structure in step 2):

The network structure of Faster RCNN is constructed in the deep learning system TensorFlow. The Faster RCNN algorithm is improved on the basis of the RCNN algorithm and the Fast RCNN algorithm. The network structure includes: input layer, pooling layer, convolution layer and output layer. Among them, the input layer is the input image, and the pooling layer compresses the input feature map. On the one hand, it makes the feature map smaller and simplifies the computational complexity of the network; on the other hand, it performs feature compression to extract the main features. Convolution performs feature extraction, and the output layer mainly refers to the fully connected layer, which is used to connect all the features and send the output value to the classifier for classification. Through the feature extraction and abstraction of the multi-level convolutional network pooling network, the detection rate of the target in the picture can be effectively improved. At the same time, the target feature can be output through the fully connected layer and classified to improve the recognition accuracy of the target in the picture. .

As shown in Figure 2, the steps to train Faster RCNN in step 3):

Step 3.1), on the already trained model (i.e. ImageNet model), train the RPN network, corresponding to stage1_rpn_train.pt, use the trained RPN network to collect proposals, corresponding to rpn_test.pt;

Step 3.2), train the Fast RCNN network for the first time, corresponding to stage1_fast_rcnn_train.pt;

Step 3.3), the second training RPN network, corresponding to stage2_rpn_train.pt, again using the trained RPN network, collecting proposals, corresponding to rpn_test.pt;

Step 3.4), train the Fast RCNN network for the second time, corresponding to stage2_fast_rcnn_train.pt.

The RPN network is used to delineate the approximate location of the target, and the Fast RCNN network performs detailed positioning and classification of the target on this basis. The above-mentioned method of sharing weights is used in the process of training the two networks, so that RPN and Fast RCNN share a part of the network. The size of the network is reduced, and the accuracy rate is improved. At the same time, because RPN uses GPU for calculation, the speed reaches the millisecond level, which is helpful for real-time detection.

As shown in Figure 3, Faster RCNN is detected in step 4):

First, use the Resnet/VGG network structure for basic feature extraction;

Secondly, the RPN network (that is, the region suggestion network) is responsible for calculating the coordinates of the area where the target may exist and judging whether it is the foreground/background and the target area (proposals) obtained by using the RPN network and then passing through the ROI Pooling layer to obtain a feature vector of the same length;

Finally, the softmax classifier is connected through two fully connected layers, in which the fully connected layer realizes specific classification and more accurate regression coordinates.

The first step can propose fewer candidate boxes, and the second step is to detect on this basis, which improves the detection efficiency and improves the detection accuracy at the same time.

As shown in Figure 4, when the person is on duty, the side with the human head on the badge should face outward (positive), indicating that someone is on duty, and the side with the human head on the badge should face inward (reverse), indicating that the person is not on duty, as shown in Table 1 .

Table 1 Personnel state discrimination logic table

The present invention replaces the selective search method with a region proposal network (RPN) to select candidate frames, and the operation speed is faster, and at the same time avoids repeated calculations when extracting features from candidate frames; the combination of the target detection network and the classification network greatly improves the overall model accuracy.

The invention innovatively divides the badge (front) and badge (reverse) into two types of labels with a balanced number of labels for training, so that the model can regard badge (front) and badge (reverse) as two different things Classification is carried out to obtain the positive and negative information of the badge.

Regarding the content disclosed in this case, the following points need to be explained:

(1) The drawings of the embodiments disclosed in this case only relate to the structure involved in the embodiments disclosed in this case, other structures can refer to the general design;

(2) In the case of no conflict, the embodiments disclosed in this case and the features in the embodiments can be combined with each other to obtain new embodiments;

The above are only the specific implementation methods disclosed in this case, but the protection scope of the present disclosure is not limited thereto, and the protection scope disclosed in this case should be based on the protection scope of the claims.

Claims (2)

1. An intelligent recognition system for the working state of personnel in an electric power business hall is characterized by comprising an acquisition module, a training module, a preprocessing module, a target detection module, a logic judgment module and an output alarm module,

the acquisition module is used for marking the video in the acquisition business hall and intercepting pictures containing effective information from the video;

the training module is used for enhancing the acquired picture data and training the depth network by using the enhanced picture data;

the preprocessing module is used for preprocessing the received pictures;

the target detection module is used for detecting targets of the images;

the logic judging module is used for judging working states of the stations and personnel in the stations through targets found in the images;

the output alarm module is used for converting the logic judgment result and providing early warning;

detecting that the working state of staff in a business hall is wrong in an image recognition mode, sending abnormal information to remind a manager to prompt staff to correct in time;

the training module comprises the following steps:

1) Selecting and labeling a sample set;

2) Constructing a Faster RCNN network structure;

3) Training Faster RCNN;

4) Detecting Faster RCNN;

selecting and marking the sample set in the step 1):

marking the sample set by using a graphic marking tool LabelImg, and manually circling each sample by using LabelImg to obtain an XML marking file of a picture corresponding to each sample data; randomly selecting a training set, a verification set and a test set from all data according to the proportion;

building a Faster RCNN network structure in the step 2):

constructing a network structure of a fast RCNN in a deep learning system TensorFlow, wherein the network structure comprises: an input layer, a pooling layer, a convolution layer and an output layer;

step 3) training the Faster RCNN:

step 3.1), training the RPN network on the trained model, corresponding to the stage1_ RPN _train. Pt, and collecting the proposals corresponding to the RPN _test. Pt by using the trained RPN network;

step 3.2), training the Fast RCNN network for the first time, and corresponding to the stage1_fast_rcnn_train. Pt;

step 3.3), the second training RPN network, corresponding to stage2_ RPN _train. Pt, re-using the trained RPN network, collecting proposals, corresponding to RPN _test. Pt;

step 3.4), training the Fast RCNN network for the second time, and corresponding to the stage2_fast_rcnn_train.

Detecting Faster RCNN in step 4):

firstly, extracting basic characteristics by utilizing a Resnet/VGG network structure;

secondly, the RPN network is responsible for calculating the coordinates of the area where the target possibly exists, judging the coordinates to be foreground/background, and obtaining the feature vector with the same length by passing through the ROI Pooling layer of the target area obtained by the RPN network;

and finally, accessing the softmax classifier through two full-connection layers, wherein the full-connection layers realize specific classification and regression coordinates.

2. The intelligent power hall personnel operating condition identification system of claim 1, wherein the targets in the target detection module comprise a display, a seat, a worker and a work board.