CN114549812A - Intelligent substation relay protection hard pressing plate checking method based on target detection model - Google Patents

Abstract

The invention discloses a target detection model-based checking method for a relay protection hard pressing plate of an intelligent substation, which specifically comprises the following steps: downloading a reference file corresponding to the work task, and establishing a connection between the two-dimension code of the scanning screen cabinet and the reference file; after the photographing is finished, the YOLOv4-tiny model is synchronously called and the reference file is analyzed, and finally, the result after the checking is displayed to the working personnel; if the conditions of missed detection and false detection do not occur, the staff can directly save the record to the local; if the phenomenon of missing detection or false detection occurs, the worker needs to correct the corresponding hard pressing plate. According to the invention, the protection screen cabinet is not required to be additionally transformed, the hard pressing plate picture shot on site is used for making a data set, a data enhancement algorithm is researched and used for expanding the data set, the problems of missing detection, error detection and the like in the prior art are further solved, and the method has higher universality and robustness.

Description

Intelligent substation relay protection hard pressing plate checking method based on target detection model

Technical Field

The invention belongs to the technical field of power grid operation and maintenance, and particularly relates to a checking method for a relay protection hard pressing plate of an intelligent substation based on a target detection model.

Background

The smart grid is a necessary trend of power grid development, as in document 1: zhangyao, Wang' ao cold, Zhang hong, China smart grid development review [ J ] electric power system protection and control, 2021,49(5):180 —. and document 2: yu Xin, urgency and longevity of smart grid implementation [ J ] Power System protection and control, 2019,47(17):1-5. The relay protection is a weak link, and particularly protects the routing inspection work of the hard pressing plate. The work is an important component of secondary operation and maintenance of power transformation, and once the conditions of misdelivery and missed delivery occur, accidents can be caused, so that the work requires high reliability, and the intelligent level of the work is difficult to improve. So that manual inspection is mainly used at the present stage, but the workload is large, the energy consumption is huge, the visual fatigue is inevitably caused, and the working efficiency is low. In order to improve the intelligent level, reduce the workload of the operation and maintenance team and avoid accidents caused by misoperation as much as possible, researchers of all parties develop research on the problems successively.

Starting from the hardware level, researchers have proposed the concept of an electric platen, as disclosed in document 3: yaoqinghua, Dongliang, Zhou Lei, etc. design and application research of intelligent remote throwing and retreating electric pressing plate [ J ] electric power system protection and control, 2015,43(20): 143-: application of intelligent pressure plate system in 500kV transformer substation of Zeqih Harvon (500 kV power system protection and control), 2010,38(23): 219-. Therefore, some researchers have proposed a method of image processing by means of computer technology, as in document 5: protective platen positioning and state recognition based on color separation and morphological feature analysis studies [ J/OL ]. electrical measurements and instrumentation 1-8[2021-03-07]. http:// kns.cnki.net/kcms/detail/23.1202.th.20201127.1043.010.html., document 6: paraffin, Tanjian, Wuxi spring, etc. intelligent substation protection pressure plate state identification based on image processing and morphological feature analysis [ J ] electric power automation equipment, 2019,39(07): 203-: the method comprises the steps of carrying out image processing-based protective pressing plate positioning and state recognition research [ J ] on digital technology and application, 2018,36(06):82-83, wherein the method needs to carry out preprocessing (generally adopts morphological processing) on an acquired image to achieve a certain standard, then, document 5 adopts color separation, document 6 adopts morphological feature analysis, and document 7 adopts a HOG + SVM mode to identify the pressing plate state, but the method has higher requirements on the image, and a plurality of thresholds are difficult to select in the standardization process. Due to the rise of artificial intelligence in recent years, there are researchers trying to find new methods from this field, in which a large number of image recognition methods are used, as in document 8: worry, ruhaojie, the handle of the Shengang, etc. application of artificial intelligence AI technology in protecting platen state identification [ J ] integrated circuit application, 2020,37(06): 122-: lujiawei, liu rui honor, identification design of the throwing and retreating states of a relay protection pressing plate based on color template matching [ J ] electromechanical information, 2018(36), 115 and 116, and 10: chengzhiqiu, penyongjian, lizheng strong, research on automatic scanning and identification technology of hard pressing plate of automatic switch for distribution network [ J ] bonding, 2020,43(07):78-81, document 11: rendu jie, jiang lan, power system relay protection pressing plate image recognition system [ J ] university journal of beijing (natural science edition), 2004(02) 60-64, document 12: dun should pine, chun qin, sons xiao pine the identification method of the protection pressing plate throwing-retreating state based on image recognition [ J ]. shanxi electric power, 2015,43(10):49-53+67. and document 13: the method comprises the steps of (1) carrying out image recognition on the basis of the state recognition technology research and application of protective pressing plates [ J ]. power equipment management, 2020(01):139- & 140. however, the first step of image recognition still needs image preprocessing, and although probabilistic Hough transformation is added in the document 9 and AdaBoost and other algorithms are used in the document 10, the recognition rate is still low. There are also researchers who choose clustering algorithms, as in document 14: protection platen state identification technology [ J ] based on model cluster matching and morphological feature recognition, shaanxi electric power, 2017,45(01):32-36+85, and document 15: the method comprises the steps of using clustering and evidence theory to realize the state check of a transformer substation protection pressure plate [ J ] power grid technology, 2020,44(06): 2343-. However, this method is a color matching method, and a color discrimination method has been tried. In the process of collecting images, the color of a plurality of hard pressing plates is found to be very close to the color of a screen cabinet, and the hard pressing plates are all similar to camel, so that the hard pressing plates are difficult to distinguish from the background. There are also methods that researchers have tried deep learning, such as: document 16: wudi, Tang soldier, Lipeng, Yang reinforcement, Wen Bo, Ri HengXuan, a transformer substation relay protection device state monitoring technology [ J ] based on a deep neural network, protection and control of an electric power system, 2020,48(05):81-85.DOI:10.19783/j.cnki.pspc.190516. a convolutional neural network + feature transformation is adopted; document 17: wangwei, yanyanglong, Dian hui, etc. the intelligent recognition of the state of the pressing plate of the transformer substation based on the OpenCV + SSD deep learning model [ J/OL ] electric measurement and instrument 1-10[2020-09-13]. http:// kns. cnki. net/kcms/detail/23.1202.TH.20200827.1838.052.html. and document 18: an improved platen state recognition SSD algorithm [ J/OL ] electrical measurement and instrumentation 1-10[2020-09-17] http:// kns.cnki.net/kcms/detail/23.1202. TH.20200917.1717.002.html.adopted SSD target detection model; document 19: the transformer substation pressure plate switch state recognition system based on the machine vision is researched by [ D ]. Wuhan university of theories, 2019. a Yolov3 target detection model is adopted, but the model precision is lower than that of an article. There are also researchers, document 20: the application of AR augmented reality technology in transformer substation secondary equipment operation and inspection [ J ] power system protection and control, 2020,48(15): 170-: schungsank, wushishi, grandson, etc. accurate identification method of protective platen state based on phase features [ J ] university of anhui proceedings (natural science edition), 2020,44(03):38-42. still further, a platen state identification method based on phase competition coding is proposed, and document 22: intelligent checking method of relay protection pressing plate [ J ] university of Chongqing, 2015,38(06):91-98 and 23: zhang Man, Feng Geng, Liu Tong, etc. 500kV intelligent substation pressure plate state monitoring and intelligent checking technology [ J ] electrician technology, 2018(24):89-90+93.

Chinese patent CN113794277A (application number: 202110980430.4) invented a platen state image recognition method and system, which is also based on deep learning and also uses mobile equipment to replace manual inspection work, but the accuracy of the deep learning model of the invention is lower than that of the present invention, and the problems of missing detection and false detection of the model are not further solved, although the missing report rate and the false report rate of the model are not higher than 3%, when such problems occur, the present invention can only repeat the operation again, and this will reduce the work efficiency.

Chinese patent CN113255827A (application number: 202110669583.7) proposes a system and a method for recognizing the state of a pressure-holding relay plate based on the YOLO Nano algorithm, and the system and the method are also based on the YOLO algorithm and a mobile device, but the mobile device in the system only completes the recognition of the state of the pressure plate, and the core check work in the inspection work is not completed in the mobile device, so the system and the method require frequent data exchange between the mobile device and a server, and the amount of data exchanged at a time is larger.

The key problems to be solved by the invention are as follows:

1) the accuracy of the recognition algorithm is insufficient. The recognition model designed in the invention with patent publication number (CN) 113794277a still has a false negative rate and false positive rate of nearly 3%, which is especially obvious when the image capturing angle is inclined or the light is not uniform. The appearance of the missing report and the false report can lead the inspection personnel to take images for many times, thereby reducing the working efficiency and solving the problem urgently.

2) False positives and false negatives are problems corrected and remedied by pressure plates. The target detection algorithm based on deep learning cannot guarantee the recognition accuracy of 100%, namely, the situations of false detection and missed detection cannot be completely eradicated, for example, the invention with the patent publication number (CN) of 113255827A can only shoot an image once again when the false detection and the missed detection occur, and then the image is detected again. Often times, false detection and missed detection cannot be solved by secondary detection, and therefore, the problem needs to be solved from other angles.

Disclosure of Invention

In order to solve the problems, the invention provides a checking method of a relay protection hard pressing plate of an intelligent substation based on a target detection model.

The invention discloses a method for checking a relay protection hard pressing plate of an intelligent substation based on a target detection model, which comprises the following steps of:

step 1: and downloading a reference file corresponding to the work task, and establishing a connection between the two-dimension code of the scanning screen cabinet and the reference file.

The working personnel need to download the reference file corresponding to the work task according to the work number of the personnel, and then start the routing inspection work; in the process of inspection, the two-dimensional code of each screen cabinet needs to be scanned first, and the scanning result is used for establishing contact with the reference file.

Step 2: data acquisition and data enhancement.

Shooting a hard pressing plate picture on site for making a data set; the pictures are marked by using a LabelImg tool, and the marked labels are two types: the 0-green frame and the 1-orange frame represent the two categories of "throw" and "retreat", respectively.

The original image is then normalized and preprocessed.

And step 3: and analyzing the reference file.

Converting the reference file into an XML file, wherein the XML file comprises a transformer substation name, a protection device identification code, a hard pressing plate name and normal operation state information; wherein x and y in the Pad label represent the hard pressing plate in the x-th row and the y-th column, and the positions which are not used are standby hard pressing plates; and the parsing of the reference file is realized by a DOM parser.

And 4, step 4: YOLOv4-tiny model call.

Selecting an Dnn module calling model using OpenCV; before calling, OpenCV is added to a project in a Module mode, and then OpenCV library files are copied to the jniLibs folder.

And 5: and (4) judging missing detection and automatically completing.

And S51, performing bubble sorting according to the y coordinate of each bounding box, traversing the whole list, and judging the position of the line needing to be changed.

S52, after the position of the line change is determined, bubble sorting is carried out in the line according to the x coordinate of the bounding box, and a two-dimensional array is formed; and in the traversal process, counting the total number of detection results, the initial coordinate of each row, the total width and the total height of all the bounding boxes and the total interval between the two bounding boxes so as to obtain the average initial coordinate of each row, the average width, the average height and the average interval of the bounding boxes.

And S53, traversing again, judging the position of the line change under the same condition as the first traversal, and judging whether the number of the hard pressing plates in the line is enough to 9 according to the principle that the number of the hard pressing plates in each line of the relay protection screen cabinet is 9, wherein the hard pressing plates in the line do not need to be complemented if the number of the hard pressing plates in the line is enough, and the hard pressing plates in the line are complemented if the number of the hard pressing plates in the line is not enough.

S54, when completing, firstly, judging whether completing is needed according to the difference value of the average initial coordinate of the line and the head coordinate of the line; rounding the quotient of the difference value and the average width of the bounding box, and judging the number of the products needing to be completed; completing the current row head position of the row from back to front; then, judging whether the row needs to be completed or not according to the difference value between the boundary frames in the row; finally, if the number of the bounding boxes of the row is still less than 9 after the row head and the row are both completed, the completion is performed backwards from the current row end position of the row.

Step 6: false detection judgment and convenient correction.

And judging that the model is false detection when the identification result of the model is inconsistent with the reference file.

When the recognition result of the model is obtained, a user needs to change the recognition result, then marks the currently shot picture, then adds the picture and the corrected result into a model training set, and uses the picture to train the model continuously, so that the generalization of the model is improved; when the reference file is a reference file, a user needs to check and modify the reference mark, and the reference is manually checked and modified; after the inspection work is finished each time, all the work data can be transmitted back to the server, and the subsequent model training and reference checking work is finished at the server side.

Further, the normalization and the pretreatment in the step 2 specifically include:

s21, the original image is normalized to 416 × 416.

S22, randomly reducing the picture and twisting the length and the width; the degree of narrowing is the random number in the interval [0.8,1], and the degree of distortion is the random number in the interval [0.5,1.5 ].

S23, the remainder after the reduction is filled with black to ensure that the picture size is 416 × 416.

S24, flipping the image with a 50% probability.

S25, Gaussian blur processing is performed according to the probability of 50%.

S26, performing color gamut distortion in the HSV color space; hue randomly rotates counterclockwise or clockwise within the interval (-Hue, Hue), Saturation and Value take random numbers in positive numbers within 100 first, and then take reciprocal according to 50% probability.

Further, the specific process of implementing model calling through the Dnn module in step 4 is as follows:

and S41, loading an OpenCV library file.

And S42, loading the files of the network configuration and the training weight by using a Dnn module, and constructing a model network.

And S43, reading in the picture and converting the picture into a Bolb standard input format available for the model.

And S44, inputting the converted picture into a network to obtain an output layer of the YOLO, and obtaining a detection result list through forward propagation.

And S45, circularly traversing the detection result list, filtering out detection results smaller than a threshold value, and obtaining a bounding box list, a category list and a confidence list which correspond to one another.

And S46, performing non-maximum value suppression and reducing the number of overlapped bounding boxes.

And S47, drawing a boundary box and storing the result picture.

Further, in step S51, it is determined that the position needs to be changed, and the determination criteria include three:

a. the top left y coordinate of the next box is larger than the bottom right y coordinate of the previous box.

b. If the bounding boxes of two rows overlap, the top left y coordinate of the next box is smaller than the bottom right y coordinate of the previous box.

c. The y coordinate of the top left corner of the next box is greater than the y coordinate of the top left corner of the previous box by the value of one box height.

Further, the configuration of the model training in step 6 is as follows: the picture size of the input network is 416 × 416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1/10 at 4800 th iteration and 5400 th iteration, respectively.

The beneficial technical effects of the invention are as follows:

1) the method fully considers the requirements on safety and reliability in power work, overcomes the defect of insufficient accuracy in the invention with the patent publication number (CN) of 113794277A, and designs and realizes a more appropriate data enhancement algorithm aiming at the problem of protecting the model training data of the hard pressing plate, thereby further improving the generalization of the model and improving the detection accuracy of the model.

2) In consideration of the problems of light reflection phenomenon of a glass cabinet door of a protection screen cabinet, edge distortion of camera imaging and the like when a mobile terminal is used for shooting in the field working process, the method researches and realizes the judgment of omission and false detection of data after model prediction, achieves the effects of automatic completion of omission of a hard pressing plate and convenient correction of false detection of the hard pressing plate, and overcomes the unavoidable omission and false detection phenomena in deep learning detection methods with patent publication numbers (CN) of 113794277A, 113255827A and the like.

Based on the reasons, the protection screen cabinet is not required to be additionally transformed by combining the wide popularization of the current mobile intelligent equipment, the problems of missing detection, error detection and the like in the prior art are solved, and the mobile intelligent equipment has higher universality and robustness.

Drawings

Fig. 1 is a technical flow chart of a checking method of a relay protection hard pressing plate of an intelligent substation.

FIG. 2 is a diagram of an example of a reference file according to the present invention.

FIG. 3 is a functional display diagram of missed-inspection completion (a is before completion and b is after completion).

Fig. 4 is a diagram showing a false detection condition.

Fig. 5 is a graph showing the model training process (a is an avg loss curve, b is a training result graph).

Fig. 6 is a scene simulation display diagram (a being an initial state, b being an intermediate state, c being a final state).

Detailed Description

The invention is described in further detail below with reference to the figures and the detailed description.

The flow of the checking method for the relay protection hard pressing plate of the intelligent substation based on the target detection model is shown in fig. 1, and a worker needs to download a reference file corresponding to the work task according to the work number of the worker, and then can start to perform inspection work. In the process of inspection, the two-dimensional code of each screen cabinet needs to be scanned first, and the scanning result is used for establishing contact with the reference file. After the photographing is finished, the calling of the YOLOv4-tiny model and the analysis of the reference file are synchronously carried out, and finally, the checked result is displayed to a worker. At the moment, if the conditions of missed detection and false detection do not occur, the staff can directly save the record to the local; if the phenomenon of missing detection or false detection occurs, the worker needs to correct the corresponding hard pressing plate without repeating the previous steps. After the work records of the screen cabinet are stored, the worker can move to the next screen cabinet, and after all the screen cabinets are inspected, all the data of the work can be uploaded to the server in the area with smooth network.

The invention relates to a method for checking a relay protection hard pressing plate of an intelligent substation based on a target detection model, which comprises the following steps:

1. data acquisition

In consideration of the actual working scene of the inspection personnel, the hard pressing plate picture acquired on the site is used for making a data set on the principle of reproducing the actual environment as much as possible. The specific recurring scenario is as follows:

shooting angle: a group of pictures are taken in a state that the mobile phone is relatively parallel to the screen cabinet, because the pictures are easier to identify. However, in the shooting process, because the heights of the hard pressing plate areas are different, the user needs to frequently squat to complete shooting, and the user experience is greatly reduced. Therefore, on the premise that the hard pressing plate and the hard pressing plate are not covered or shielded, a group of pictures are shot again in a posture of not squatting.

The problem of illumination: initially, a set of pictures was taken in natural light and light respectively, but in both cases there was a problem of light reflection. In continuing attempts, it was found that the degree of light reflection can be greatly reduced when taking pictures using the flash of a cell phone, and a set of pictures is taken again using the flash point.

After the picture is collected, the picture is marked by using a LabelImg tool, and the marked labels are two types: 0 (green frame) and 1 (orange frame), representing "throw" and "retreat" categories, respectively.

2. Data enhancement

Data sets are important in deep learning, and without enough data sets as supports, the model is easy to have an overfitting problem. In other words, if the data set is insufficient, the model is likely to learn many unnecessary features. For example, when identifying a computer, the model may be considered as: an object not only has the appearance of a computer, but also needs to be a computer under a certain angle and a certain brightness. Obviously, the data enhancement can enrich and diversify the data set, so that the model can achieve the purpose of 'knowledge and universality'.

The data enhancement specifically comprises:

(1) the artwork is normalized to a size of 416 x 416.

(2) Randomly reducing the picture and performing length and width distortion; the degree of narrowing is a random number within the interval [0.8,1], and the degree of distortion is a random number within the interval [0.5,1.5 ].

(3) The remainder after the reduction is filled with black to ensure that the picture size is 416 × 416.

(4) The image is flipped with 50% probability.

(5) Gaussian blur processing was performed with a probability of 50%.

(6) Performing gamut warping in an HSV color space; hue randomly rotates counterclockwise or clockwise within the interval (-Hue, Hue), Saturation and Value take random numbers in positive numbers within 100 first, and then take reciprocal according to 50% probability.

3. Reference file design and parsing

The inspection work is generally completed by matching two persons, and one person is specially responsible for sequentially rechecking the on-off states of the reference document and the on-site protective hard pressing plate. In order to reduce the complexity of the article and save human resources, the article converts the reference file into an XML file to realize paperless operation. The operation reference of the protective hard pressing plate can be obtained by analyzing the XML file, and the XML file is compared with the model prediction result, so that the inspection and check work can be completed.

The XML format of the reference file is shown in fig. 2, and includes information such as a substation name, a protection device id, a hard platen name, and a normal operation state. Wherein x and y in the Pad label represent the hard press plate in the x row and y column, and the position not available is the spare hard press plate.

The reference file is downloaded from the server side before each work, because all operations are finished off-line in the whole inspection work process. Parsing the reference file is accomplished through a DOM parser. The essence of DOM parsing is to assemble data into a "tree", and then parse the XML file through the relationship between nodes, which is not very difficult, but rather cumbersome.

4. Model building and calling

Resource files of the Android system are roughly divided into two types: the compilable resource files under the res directory and the native resource files under the assets directory. Files under the assets folder are not precompiled like XML and Java files, but always keep the original file format, so that training files (including weight files, configuration files and the like) of the model are selected to be embedded in the assets folder. In addition, an absolute path of the training file is needed in the calling process of the model, but the file in the assets folder cannot acquire the absolute path. The article therefore takes a pre-loaded policy: when the APP is started for the first time, the training file needs to be loaded into the cache folder with the same name as the APP by a byte stream method, so as to obtain the absolute path of the training file.

After model building is completed, the article chooses to call the model using the Dnn module of OpenCV, and OpenCV has a version of 3.4.11 (to ensure good support of OpenCV on YOLOv4 series, its version is tested to be not lower than 3.4.11). OpenCV is added to the project in Module before invocation. And copying the OpenCV library file to the jniLibs folder. The specific process for realizing the model calling through the Dnn module is as follows:

(1) loading an OpenCV library file;

(2) loading a network configuration and training weight file by using an Dnn module to construct a model network;

(3) reading in a picture and converting the picture into a Bolb standard input format available for a model;

(4) inputting the converted picture into a network to obtain an output layer of the YOLO, and obtaining a detection result list through forward propagation;

(5) circularly traversing the detection result list, and filtering out detection results smaller than a threshold value to obtain a bounding box list, a category list and a confidence list which are in one-to-one correspondence;

(6) carrying out non-maximum value inhibition and reducing the number of overlapped bounding boxes;

(7) and drawing a bounding box and storing a result picture.

5. Missing detection judgment and automatic completion

In the actual test process, due to factors such as reflection and shielding, the condition of missed detection still occurs, and at the moment, the code completion is required to be passed through at the later stage.

In the model prediction, a target area, i.e. an area within each box (generally called a bounding box in the current detection) in fig. 3(a) is predicted, and then a prediction is made on the target category of the target area, which is represented by a rectangle in the figure to represent "open" and a circle to represent "closed". After the prediction is completed, the coordinates of each vertex of the bounding box and the category of the target in the bounding box can be obtained, but the disordered bounding box is obtained, and whether the detection is missed or not needs to be judged according to the coordinates. Through continuous tests and improvements, the current missed inspection completion function is realized as follows:

firstly, bubble sorting is carried out according to the y coordinate of each bounding box, so that the detection results in the same row are close together;

then, traversing the whole list, and judging the positions of the lines to be changed, wherein three judgment standards are provided, and the three judgment standards are in an OR relationship:

(1) the y coordinate of the upper left corner of the next frame is larger than the y coordinate of the lower right corner of the previous frame;

(2) if the bounding boxes of two lines overlap (due to the excessive inclination angle of the shooting angle), the y coordinate of the upper left corner of the next frame is smaller than the y coordinate of the lower right corner of the previous frame, but the value is not too large and is temporarily smaller than a fixed value;

(3) further or perhaps more specifically, it is determined that the y coordinate of the top left corner of the next box is greater than the y coordinate of the top left corner of the previous box by the height of the box, but because the box height is not constant, this is tentatively the average box height 3/4.

After the position of the line change is determined, bubble sorting is carried out in the line according to the x coordinate of the bounding box, and a two-dimensional array is formed. Counting the total number of detection results, the initial coordinate of each row, the total width and the total height of all the bounding boxes and the total interval between the two bounding boxes in the traversal process so as to obtain the average initial coordinate of each row, the average width, the average height and the average interval of the bounding boxes;

and finally, traversing again, judging the position of the line change under the same condition as the first traversal, and judging whether the number of the hard pressing plates in each line is enough to 9 according to the principle that each line of the relay protection screen cabinet is 9, if so, the completion is not needed, and if not, the completion is carried out.

When in completion, whether completion is needed or not is judged according to the difference value of the average initial coordinate of the line and the head coordinate of the line. And rounding the quotient of the difference value and the average width of the bounding box, and judging the number needing to be complemented. And completing from the current head position of the line to the front position from the back. And then judging whether the row needs to be completed or not according to the difference value between the boundary frames in the row. Finally, if the number of the bounding boxes of the row is still less than 9 after the row head and the row are both completed, the completion is performed backwards from the current row end position of the row.

The drawing method of the bounding box during completion is as follows: firstly, adding an average interval to a starting point x coordinate to serve as a starting point x coordinate of a first boundary frame; the coordinate of the starting point plus the average width is taken as the coordinate of the end point x; setting the starting point y coordinate as the starting point y coordinate of the first bounding box; adding the average height to the y coordinate of the starting point to be used as a y coordinate of the end point; this results in a complete bounding box. The same applies to the drawing of the remaining bounding boxes. FIG. 3(b) is a complementary effect diagram, which simulates the case of glistenings using Photoshop

6. False detection determination and convenient correction

The false detection means a model recognition error, but the model itself cannot determine whether or not it recognizes an error, and therefore data for comparison with this, that is, a hard platen reference, is required. When the recognition result of the model is not consistent with the reference, the program judges that the model is false detection.

Therefore, the false detection is caused by two reasons, namely, model identification error and reference file error. In the former case, the user needs to change the recognition result, then marks the current shot picture, then adds the picture and the corrected result into the model training set, and uses the picture to train the model continuously, thereby improving the generalization of the model. In the latter case, the user needs to align fiducial markers, which are then modified by manual re-checking. After the inspection work is finished each time, all the work data can be transmitted back to the server, and the subsequent model training and reference checking work is finished at the server side.

So far, there are three frames in the figure, which are respectively the frame for correct model identification, the frame for completion after missing detection and the frame for false detection, and the three sources are respectively distinguished by using three colors of green, blue and red tables. The user needs to complement and misdetect the frames.

In fig. 4, the missing detection is reproduced in the same manner as in the previous section. However, the existing model has high precision, the situation of recognition error is difficult to reproduce, and the reference file is wrongly changed at the place so as to achieve the same false detection effect.

When the user corrects, the user clicks any position in the boundary box in the screen, the APP pops up a prompt box, the hard pressing board switching state of the currently selected frame is displayed, and a change option is provided. The realization of the function needs rewriting the Android touch event, namely an onTouch method, firstly, a screen coordinate clicked by a user can be conveniently obtained in the method, then, a getImagematrix method of an ImageView control (a control used for displaying a hard pressing plate picture) is called to obtain a picture matrix, then, an inverse matrix of the picture matrix is obtained, the actual coordinate in the picture after the screen coordinate conversion is obtained through mapping of the inverse matrix, further, the position in which boundary frame can be judged, then, a prompt box is popped up to allow the user to select the state needing to be changed, finally, OpenCV is called to redraw the boundary frame according to a user selection result returned by the prompt box, and therefore, one-time false detection and correction are completed. After all the bounding boxes are corrected, other colors are all eliminated, and the visual fatigue can be relieved to a great extent by means of color distinguishing.

7. Model training

The machine configuration used for training the model is as follows:

the system comprises the following steps: windows 10;

CPU：

Core^TMi7-9700K；

GPU：NVIDIA RTX 2080 SUPER.

the hyper-parameter configuration for model training is shown in table 1:

TABLE 1 training parameter table

The picture size of the input network is 416 × 416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1/10 at 4800 th iteration and 5400 th iteration, respectively. The trend of the average loss (avg loss) of the model during the training process is shown in fig. 5(a), and the final avg loss is less than 1 and is 0.6771. The mAP of the model shown in FIG. 5(b) reached 99.44%.

TABLE 2 comparison of model Properties

Table 2 compares the article model in detail with the SSD object detection model used in document [16], and the improved SSD algorithm model used in document [17 ]. However, the author of the YOLOv3 target detection model used in the document [18] only gives the model recognition rate of 98.95%, does not give the definition and calculation method of the recognition rate, and does not give other indexes, so table 2 does not compare. And the article previously used YOLOv3 and tested the YOLOv3 model to an accuracy of only 95.45%.

After the training was completed, an actual test was performed using a three-star Note9 smartphone (CPU is high-pass cellulon 845, operating the memory 6GB, and the overall performance belongs to the middle-end queue). The article randomly selects 25 pictures in a test set, the identification accuracy rate of YOLOv4-tiny is 98.64%, and the identification speed is about 1 s. Table 3 details the results of the YOLOv4-tiny model test.

Table 3 mobile phone test meter

8. Working environment simulation display

Supposing that the shot picture has strong reflection at the hard pressing plates in the first row and the fourth column, so that the hard pressing plates cannot be identified by the model at the position, and the phenomenon of missing detection occurs; the reference file is also modified to simulate the false detection phenomenon. Then the initial state is as shown in fig. 6(a), the bounding box of the fourth row and column (since the color has been changed to black in the figure, the specific position of the bounding box is specifically indicated) is blue, and the bounding box is completed; the bounding box of the first row and the fifth column is red and is a false detection bounding box. Clicking the positions of the blue frame and the red frame respectively, the APP pops up a prompt box, as shown in fig. 6 (b). The prompt box will display the click location and the current state of the hard platen at that location and provide a change option. When the changed state selected by the user is different from the current state, the bounding box is redrawn, as shown in fig. 6(c), and the shapes of the two bounding boxes are changed, and the color is changed into green.

After the modification, all the bounding boxes are changed into green, so that the operation result of the user on the screen cabinet can be stored.

The use of the boundary frames with different colors to distinguish different conditions is helpful for improving the identification degree, and the operation is very simple, thereby not only being helpful for reducing the accident rate, but also greatly improving the working efficiency.

The method applies a YOLOv4-tiny model to a mobile intelligent terminal, completes the state identification of a protective hard pressing plate under the support of OpenCV, designs and uses a reference file in an XML format, compares the analyzed reference file with a model prediction result, and assists in a missed inspection completion and false inspection correction strategy, so that the mobile terminal can replace manual work to complete inspection work.

Because the protective hard pressing plate mainly has two types of a compression type and a plug-in type, the structure is simpler, and the switching state is relatively easy to identify, the article provides a novel relay protection hard pressing plate intelligent checking technology based on a light-weight and quick YOLOv4-tiny target detection model.

The Average Precision (mAP) of the model Mean value reaches 99.44%, and the Precision rate, the recall rate and the F1 score are all 0.98. Under the support of OpenCV, the state identification and inspection check work of the protective hard pressing plate can be completed at the mobile terminal. Although the model precision reaches a high standard, the missing detection and the false detection of the model are still unavoidable in the actual testing process. Therefore, the invention realizes the automatic completion of the missed detection hard pressing plate (short for missed detection completion), the user can directly correct the false detection hard pressing plate (short for false detection correction) by clicking the screen of the mobile equipment, and the two strategies are assisted to realize that the user can finish the inspection work of one protection screen cabinet only by shooting once, thereby greatly improving the working efficiency, ensuring that the protection hard pressing plate data at the mobile end is easy to use across platforms, and effectively improving the intelligent operation and maintenance level of relay protection.

Claims (5)

1. A method for checking a relay protection hard pressing plate of an intelligent substation based on a target detection model is characterized by comprising the following steps:

step 1: downloading a reference file corresponding to the work task, and establishing a connection between the two-dimension code of the scanning screen cabinet and the reference file;

the working personnel need to download the reference file corresponding to the work task according to the work number of the personnel, and then start the routing inspection work; in the process of inspection, firstly scanning the two-dimensional code of each screen cabinet, and using the scanning result to establish contact with a reference file;

step 2: data acquisition and data enhancement;

shooting a hard pressing plate picture on site for making a data set; the pictures are marked by using a LabelImg tool, and the marked labels are two types: a 0-green frame and a 1-orange frame, which respectively represent two categories of 'throw' and 'retreat';

then normalizing and preprocessing the original image;

and step 3: analyzing the reference file;

converting the reference file into an XML file, wherein the XML file comprises a transformer substation name, a protection device identification code, a hard pressing plate name and normal operation state information; wherein x and y in the Pad label represent the hard pressing plate in the x-th row and the y-th column, and the positions which are not used are standby hard pressing plates; the analysis of the reference file is realized through a DOM (document object model) analyzer;

and 4, step 4: YOLOv4-tiny model call;

selecting an Dnn module calling model using OpenCV; adding OpenCV to the project in a Module mode before calling, and copying an OpenCV library file to a jniLibs folder;

and 5: judging missing detection and automatically completing;

s51, performing bubble sorting according to the y coordinate of each bounding box, traversing the whole list, and judging the position of the line needing to be changed;

s52, after the position of the line change is determined, bubble sorting is carried out in the line according to the x coordinate of the bounding box, and a two-dimensional array is formed; counting the total number of detection results, the initial coordinate of each row, the total width and the total height of all the bounding boxes and the total interval between the two bounding boxes in the traversal process so as to obtain the average initial coordinate of each row, the average width, the average height and the average interval of the bounding boxes;

s53, traversing again, judging the position of the line change under the same condition as the first traversal, and judging whether the number of the hard pressing plates in each line is enough to 9 according to the principle that the number of the hard pressing plates in each line of the relay protection screen cabinet is 9, wherein the hard pressing plates in each line do not need to be complemented if the number of the hard pressing plates in each line is enough, and the hard pressing plates in each line are complemented if the number of the hard pressing plates in each line is not enough;

s54, when completing, firstly, judging whether completing is needed according to the difference value of the average initial coordinate of the line and the head coordinate of the line; rounding the quotient of the difference value and the average width of the bounding box, and judging the number of the products needing to be completed; completing the current row head position of the row from back to front; then, judging whether the row needs to be completed or not according to the difference value between the boundary frames in the row; finally, if the number of the bounding boxes of the line is still less than 9 after the completion of the head and the line of the line, the completion is carried out backwards from the current line end position of the line;

step 6: misdetection judgment and convenient correction;

when the recognition result of the model is inconsistent with the reference file, judging that the model is false detection;

when the recognition result of the model is obtained, a user needs to change the recognition result, then marks the currently shot picture, then adds the picture and the corrected result into a model training set, and uses the picture to train the model continuously, so that the generalization of the model is improved; when the reference file is a reference file, a user needs to check and modify the reference mark, and the reference is manually checked and modified; after the inspection work is finished each time, all the work data can be transmitted back to the server, and the subsequent model training and reference checking work is finished at the server side.

2. The method for checking the relay protection hard pressing plate of the intelligent substation based on the target detection model according to claim 1, wherein the normalization and the preprocessing in the step 2 specifically comprise:

s21, normalizing the original image to 416 × 416 size;

s22, randomly reducing the pictures and twisting the length and the width; the reduction degree is the random number in the interval [0.8,1], and the distortion degree is the random number in the interval [0.5,1.5 ];

s23, filling the rest part after being reduced with black, and ensuring that the picture size is 416 multiplied by 416;

s24, overturning the image according to the probability of 50%;

s25, performing Gaussian blur processing according to the probability of 50%;

s26, performing color gamut distortion in the HSV color space; hue randomly rotates counterclockwise or clockwise within the interval (-Hue, Hue), Saturation and Value take random numbers in positive numbers within 100 first, and then take reciprocal according to 50% probability.

3. The method for checking the relay protection hard pressing plate of the intelligent substation based on the target detection model according to claim 1, wherein the specific process of realizing model calling through the Dnn module in the step 4 is as follows:

s41, loading an OpenCV library file;

s42, loading files of network configuration and training weight by using a Dnn module, and constructing a model network;

s43, reading in the picture and converting the picture into a Bolb standard input format available for the model;

s44, inputting the converted picture into a network to obtain an output layer of the YOLO, and obtaining a detection result list through forward propagation;

s45, circularly traversing the detection result list, filtering out detection results smaller than a threshold value, and obtaining a bounding box list, a category list and a confidence list which correspond to one another one by one;

s46, carrying out non-maximum suppression and reducing the number of overlapped bounding boxes;

and S47, drawing a boundary box and storing the result picture.

4. The method for checking the relay protection hard pressure plate of the intelligent substation based on the target detection model of claim 1, wherein the position where line replacement is required is judged in step S51, and the judgment criteria include three:

a. the y coordinate of the upper left corner of the next frame is larger than the y coordinate of the lower right corner of the previous frame;

b. if the bounding boxes of two rows overlap, the y coordinate of the upper left corner of the next box is smaller than the y coordinate of the lower right corner of the previous box;

c. the y coordinate of the top left corner of the next box is larger than the y coordinate of the top left corner of the previous box by the value of the height of one box.

5. The method for checking the relay protection hard pressure plate of the intelligent substation based on the target detection model in claim 1, wherein the model training in the step 6 is configured to: the picture size of the input network is 416 × 416, the initial learning rate is 0.00261, the total number of iterations is 6000, and the learning rate is reduced by 1/10 at 4800 th iteration and 5400 th iteration, respectively.

Images