CN114998271B - A substation high-voltage switchgear secondary warehouse panel image recognition system - Google Patents

Abstract

The present invention relates to the technical field of substation high-voltage switchgear image recognition technology, in particular to an image recognition system for substation high-voltage switchgear secondary warehouse panels. , collect the panel pictures of the secondary warehouse, and select a clear picture for each high-voltage switchgear as a processing template picture; secondly, divide each template picture into the indicator light area, the pressure plate switch area, and the meter area, and cut out Mark each component with a component rectangle on the region image; then, detect the AprilTag label in the template image and non-template image, obtain the indicator light and pressure plate switch data set, and the digital table data set; train the ResNet50 classification network again to obtain the indicator light and pressure plate For switch status, an automatic reading method based on template matching is designed to obtain digital meter readings; finally, according to the standard threshold value of each component in the normal operating state, it is logically judged whether the component is abnormal, and an alarm is issued in time.

Description

An image recognition system for secondary compartment panels of high-voltage switch cabinets in substations

Technical Field

The present invention relates to the technical field of image recognition of high-voltage switch cabinets in substations, and in particular to an image recognition system for secondary compartment panels of high-voltage switch cabinets in substations.

Background Art

For daily inspection of high-voltage switchgear in substations, on-site staff check the secondary compartment panel of the high-voltage switchgear and record the status of each component, so as to obtain the status of the switchgear in the primary compartment of the high-voltage switchgear. However, at present, the inspection of high-voltage switchgear in most substations mainly relies on manual work. The status of components on each side of the high-voltage switchgear is recorded through regular manual inspection. Once the switchgear failure cannot be discovered in time, it may cause serious losses. However, since the secondary compartment panel of the high-voltage switchgear contains a large number of components, it is necessary to simultaneously identify the indicator light status, the pressure plate switch status and the digital meter reading, which is difficult to achieve directly through a single deep learning model. In addition, the deep learning model can only obtain good recognition effects after training on a large amount of labeled data. At the same time, the image recognition algorithm based on deep learning has the problems of missed detection of components and misdetection of non-components as components in actual application, resulting in the recognition results not corresponding to the components in the actual position. Therefore, it is impossible to achieve reliable application of image recognition of the secondary compartment panel of the high-voltage switchgear by only using algorithms based on deep learning. It is urgent to design a universal image recognition system with high recognition accuracy, which can simultaneously identify different types of components and make it valuable for promotion and application.

Summary of the invention

The purpose of the present invention is to solve the shortcomings of the prior art and solve the problem that high-voltage switch cabinets rely on manual inspections and free workers' hands, thereby proposing an image recognition system for secondary compartment panels of high-voltage switch cabinets in substations.

In order to achieve the above object, the present invention adopts the following technical solution: a substation high-voltage switch cabinet secondary compartment panel image recognition system, comprising the following steps:

1) On the horizontal center axis of the secondary compartment panel of each high-voltage switch cabinet, post an AprilTag label with a unique ID, and record the one-to-one correspondence between the high-voltage switch cabinet name and the label ID;

2) Collect sample images of high-voltage switchgear to form a sample library. Each sample image contains the secondary compartment panel and AprilTag label of the substation high-voltage switchgear;

3) Select one clear picture from the sample library as a template picture for each high-voltage switchgear cabinet;

4) Call the AprilTag algorithm to detect all tag IDs and the pixel coordinates of the four corner points of the tags in the template image, and calculate the difference between the horizontal pixel coordinates of the center point of the tag and the horizontal central axis pixel coordinates of the sample image. The one with the smallest difference is the corresponding tag on the current high-voltage switchgear;

5) According to the corresponding tag ID obtained in step 4), search the name and ID list in step 1), obtain the name of the high-voltage switchgear collected by the template image, and store the template image name, tag ID and four corner point pixel coordinates in the database;

6) According to the component types and distribution characteristics on the secondary compartment panel of the high-voltage switch cabinet, each template image is divided into the indicator light area, the pressure plate switch area, and the digital meter area, and the regional images are cut and named;

7) Mark the components in the area image, name the components, cut the component images one by one according to the component marking frame, and set the standard state of each component;

8) Read non-template images from the sample library in sequence, and repeat step 4) to obtain the label ID and four corner pixel coordinates in each non-template image;

9) Calculate the affine transformation matrix between the template image and the non-template image based on the pixel coordinates of the label corner points in step 8) and the pixel coordinates of the label corner points in the template image in step 4);

10) According to the affine transformation matrix, the area annotation box in the template image is mapped to the non-template image, the area image is cut and named and saved, and then all the component annotation boxes in the template image are mapped to the non-template image to realize the positioning of the non-template image components;

11) According to the located component annotation box, cut the component image of each non-template image in the sample library, name each component image, and obtain the component image dataset;

12) Divide the data set into an indicator light data set, a pressure plate switch data set, and a digital table data set according to the component image names;

13) Based on the indicator light and pressure plate switch data set obtained in step 12), data augmentation is performed, the data set is divided into a training set, a validation set, and a test set, the data set and the ResNet50 classification model are imported on the algorithm server, and the indicator light and pressure plate switch classification algorithm model based on ResNet50 is trained to obtain the "on" or "off" state of the indicator light and the "open" or "closed" state of the pressure plate switch;

14) Based on the digital table data set obtained in step 12), an automatic reading algorithm based on template matching is designed to produce a binary template image of LED digital numbers from 0 to 9, and the pixel matching degree between the single binary image of the LED digital table and the digital binary template image in the new image is calculated one by one, and the matching degree is sorted to obtain the single digital reading with the highest matching degree, and the digital positions are sorted according to the horizontal coordinates, and the actual reading of the digital table is calculated according to the number of digits;

15) Input the newly acquired high-voltage switchgear image into the high-voltage switchgear image recognition system, repeat steps 4) and 5) to obtain the pixel coordinates of the four corner points of the label in the image, and repeat step 6) to obtain the regional image;

16) Calculate the affine transformation matrix of the newly collected image and the template image, map all component annotation boxes in the template image to the newly collected image, cut the component image and name it, and call the classification algorithm model based on ResNet50 or the automatic reading algorithm based on template matching according to the component type in the component image name to obtain the recognition result;

17) Read the preset standard status of each indicator light, pressure plate switch, and digital meter, logically determine whether the component status is abnormal, and issue an alarm in time.

As a further description of the above technical solution:

The tag ID in step 1) is specifically a natural number other than 0.

As a further description of the above technical solution:

The component types on the secondary warehouse panel in step 6) mainly include LED indicator lights, pressure plate switches, and LED digital meters, which are named LED, SWITCH, and METER in sequence; the divided areas in step 6) are divided by regional rectangular frames, and each high-voltage switch cabinet secondary warehouse panel image is divided into up to 3 areas, and the areas are named "*P-PSx" in sequence, where "*P" represents the abbreviation of the high-voltage switch cabinet, "PSx" represents the abbreviation of the area, and "x" takes the values of 1, 2, and 3, and the values are not repeated; the regional rectangular frame is required to frame all the components to be identified according to the component type, and use the area name as the key value, and the pixel coordinates of the upper left and lower right corners of the regional rectangular frame as the value, which are stored in the Redis database; the cutting area image in step 6) cuts the picture according to the regional rectangular frame, and after cutting, the regional image is named "*P-PSx-time.jpg" according to the regional name, "time" is the current number of milliseconds on the computer, and the regional image is saved in the local server; the operating system of the local server is LinuxUbuntu.

As a further description of the above technical solution:

The labeled components of step 7) are labeled with component rectangular frames, each component is named, and the standard state of each component under the normal operating state of the high-voltage switch cabinet is set, and the coordinates of the corner points of the component labeling frame and the component name are stored in the database. The component naming is combined with the area name, component type and number. The specific naming rules of the three components are "*P-PSx-LEDy", "*P-PSx-SWITCHy", and "*P-PSx-METERy". "y" represents a non-zero natural number. In a single area image, y has a unique value. The storage component labeling frame corner point coordinates and component name, with the component name as the key value, the component labeling frame and the component standard state in the area image as the value, are stored in the Redis database; the cutting component image of step 7) is named for each cut component image, and the component image is named "*P-PSx-LEDy-time.jpg" or "*P-PSx-SWITCHy-time.jpg" or "*P-PSx-METERy-time.jpg" according to the component name, and the component image is saved in the local server.

As a further description of the above technical solution:

The calculation of the affine transformation matrix in step 9) is specifically:

Among them, M _2×3 is the affine transformation matrix, which means that the pixel homogeneous coordinates (x ₀ ,y ₀ ,1) in the template image are transformed to the pixel homogeneous coordinates (x ₁ ,y ₁ ,1) at the corresponding position in the non-template image through linear transformation. M can be solved by the corresponding three pairs of pixel coordinates in the template image and the non-template image;

The three pairs of pixel coordinates are selected as the pixel coordinates of the upper left, upper right and lower right corner points of the AprilTag label, and the affine transformation matrix can be solved by substituting them into the above formula.

As a further description of the above technical solution:

The non-template image component positioning in step 10) is specifically as follows:

A1: Read all data with the key "*P" from the Redis database, traverse and compare the tag ID value in the value with the tag ID value detected in the current non-template image, and when the IDs are equal, obtain the name of the high-voltage switchgear corresponding to the current non-template image;

A2: Read all data with the key "*P-PSx" from the Redis database, where "*P" is the name of the high-voltage switchgear corresponding to the current non-template image. Substitute the pixel coordinates of the upper left and lower right corners of the template image's area rectangle into the following formula to obtain the coordinates of the non-template image's area rectangle. Cut the non-template image according to the transformed area rectangle.

A3: Read all data with the key "*P-PSx-LEDy", "*P-PSx-SWITCHy", and "*P-PSx-METERy" from the Redis database, substitute the pixel coordinates of the upper left and lower right corners of the template image component rectangle into the above formula to obtain the coordinates of the component rectangle of the non-template image, and locate the component of the non-template image area image according to the transformed component rectangle.

As a further description of the above technical solution:

The step 14) of making the LED digital binary template image of 0 to 9 is specifically as follows:

B1: Collect multiple digital table component images containing digits 0 to 9, perform image grayscale processing and binarization, and set the binarization threshold to 150, that is, if the grayscale value of the pixel is greater than 150, it is set to 255, otherwise it is set to 0;

B2: Use the cv2.findContours method in OpenCV to find the contour of the binary image and obtain the contour points [(x ₂ ,y ₂ ),...,(x _n ,y _n )]. Then, traverse and compare the maximum and minimum values of x ₁ to x _n and y ₁ to _yn to obtain the upper left corner pixel coordinates (x _min, y _min ) and lower right corner pixel coordinates (x _max ,y _max ) of the minimum rectangle surrounding the digital LED.

B3: Cut the digital table component image according to the four parameters (x, y, w, h), the parameter values are:

B4: Reset the image size of the digital binary template image to (42,51), save the digital template image, and name it "num.jpg" according to the actual meaning of the number. The value range of "num" is a natural number from 0 to 9.

As a further description of the above technical solution:

The calculation of pixel matching degree in step 14) reads the single digital binary template image in sequence from numbers 0 to 9 according to the picture name, and calculates the matching degree between the current single digital binary image and the template image:

Among them, _b0 is the binary pixel matrix of the current single digital image, and _b1 is the binary pixel matrix of the read-in template image.

As a further description of the above technical solution:

The matching degree sorting in step 14) adopts the bubble sorting method to obtain the maximum matching degree. The number corresponding to the binary template image is the reading result of the current single number and is inserted into the result list.

As a further description of the above technical solution:

The actual reading of the digital meter in step 14) is calculated according to the meaning of the digit size:

Among them, l is the length of the result list, that is, the number of detected numbers, and result(k) is the kth element of the result list.

The present invention has the following beneficial effects:

Compared with the prior art, the image recognition system for the secondary compartment panel of the high-voltage switch cabinet of the substation has a high recognition accuracy compared with the previous image recognition algorithm for the secondary compartment panel of the high-voltage switch cabinet, and is universal for other types of high-voltage switch cabinets. Only one ResNet50 classification model consumes GPU computing power, so it also has the advantage of low computing power consumption; according to the method of the present invention, for each secondary compartment panel of the high-voltage switch cabinet, only 5 to 10 images need to be collected to realize algorithm design and training, which solves the problem of dependence on a large amount of data; according to the method of the present invention, for each secondary compartment panel image of the high-voltage switch cabinet, all components only need to be marked once, so that the components in each image can be accurately positioned, which solves the problem of cumbersome marking data; according to the characteristics of the secondary compartment panel of the high-voltage switch cabinet of the substation, the present invention selects a combination of traditional image processing and deep learning models to simultaneously identify the indicator light, pressure plate switch, and digital meter problem in the entire image, which is simplified to the classification problem of the indicator light and the pressure plate switch, and the reading problem of the digital meter, making up for the defects of the single image processing method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a picture of a template selected for a 12-sided high-voltage switch cabinet of the present invention;

FIG2 is a diagram of the image area division of 4P and 6P high-voltage switchgears according to the present invention;

FIG3 is a diagram showing the results of cutting the image area of a 14P high-voltage switch cabinet according to the present invention;

FIG4 is a diagram showing the component labeling results in the 14P high-voltage switchgear area image according to the present invention;

FIG5 is a diagram of indicator light data set and classification obtained by the present invention;

FIG6 is a diagram of a pressure plate switch data set and classification obtained by the present invention;

FIG7 is a loss curve of the present invention in the training of the ResNet50 classification algorithm model;

FIG8 is a flowchart of the automatic reading algorithm of the digital meter of the present invention.

DETAILED DESCRIPTION

1-8, a substation high-voltage switch cabinet secondary compartment panel image recognition system provided by the present invention includes the following steps:

1) On the horizontal center axis of the secondary compartment panel of each high-voltage switch cabinet, post an AprilTag label with a unique ID, and record the one-to-one correspondence between the high-voltage switch cabinet name and the label ID;

2) Collect sample images of high-voltage switchgear to form a sample library. Each sample image contains the secondary compartment panel and AprilTag label of the substation high-voltage switchgear;

3) Select one clear picture from the sample library as a template picture for each high-voltage switchgear cabinet;

4) Call the AprilTag algorithm to detect all tag IDs and the pixel coordinates of the four corner points of the tags in the template image, and calculate the difference between the horizontal pixel coordinates of the center point of the tag and the horizontal central axis pixel coordinates of the sample image. The one with the smallest difference is the corresponding tag on the current high-voltage switchgear;

5) According to the corresponding tag ID obtained in step 4), search the name and ID list in step 1), obtain the name of the high-voltage switchgear collected by the template image, and store the template image name, tag ID and four corner point pixel coordinates in the database;

6) According to the component types and distribution characteristics on the secondary compartment panel of the high-voltage switch cabinet, each template image is divided into the indicator light area, the pressure plate switch area, and the digital meter area, and the regional images are cut and named;

7) Mark the components in the area image, name the components, cut the component images one by one according to the component marking frame, and set the standard state of each component;

8) Read non-template images from the sample library in sequence, and repeat step 4) to obtain the label ID and four corner pixel coordinates in each non-template image;

9) Calculate the affine transformation matrix between the template image and the non-template image based on the pixel coordinates of the label corner points in step 8) and the pixel coordinates of the label corner points in the template image in step 4);

10) According to the affine transformation matrix, the area annotation box in the template image is mapped to the non-template image, the area image is cut and named and saved, and then all the component annotation boxes in the template image are mapped to the non-template image to realize the positioning of the non-template image components;

11) According to the located component annotation box, cut the component image of each non-template image in the sample library, name each component image, and obtain the component image dataset;

12) Divide the data set into indicator light data set, pressure plate switch data set, and digital table data set according to the component image name; specifically, the indicator light data can be divided into two categories, namely, indicator light on and off, respectively recorded as LED_on and LED_off, and the digital table data set is the LED digital table showing different readings;

13) Based on the indicator light and pressure plate switch data set obtained in step 12), data augmentation is performed, and the data set is divided into a training set, a validation set, and a test set. The data set and the ResNet50 classification model are imported on the algorithm server, and the indicator light and pressure plate switch classification algorithm model based on ResNet50 is trained to obtain the "on" or "off" state of the indicator light and the "open" or "closed" state of the pressure plate switch; specifically: the indicator light and pressure plate switch data set is a mixture of the indicator light data set and the pressure plate switch data set, and is divided into four types of component states: LED_on, LED_off, SWITCH_on, and SWITCH_off;

14) Based on the digital table data set obtained in step 12), an automatic reading algorithm based on template matching is designed to produce a binary template image of LED digital numbers from 0 to 9, and the pixel matching degree between the single binary image of the LED digital table and the digital binary template image in the new image is calculated one by one, and the matching degree is sorted to obtain the single digital reading with the highest matching degree, and the digital positions are sorted according to the horizontal coordinates, and the actual reading of the digital table is calculated according to the number of digits;

15) Input the newly acquired high-voltage switchgear image into the high-voltage switchgear image recognition system, repeat steps 4) and 5) to obtain the pixel coordinates of the four corner points of the label in the image, and repeat step 6) to obtain the regional image;

16) Calculate the affine transformation matrix of the newly collected image and the template image, map all component annotation boxes in the template image to the newly collected image, cut the component image and name it, and call the classification algorithm model based on ResNet50 or the automatic reading algorithm based on template matching according to the component type in the component image name to obtain the recognition result;

17) Read the preset standard status of each indicator light, pressure plate switch, and digital meter, logically determine whether the component status is abnormal, and issue an alarm in time; specifically:

For indicator lights and pressure plate switches, when the standard state and the identification state are the same, it is judged as normal, and when the standard state and the identification state are different, it is judged as abnormal;

For a digital meter, when the identification reading is within the preset threshold range, it is judged as normal, and when the identification reading is greater than or less than the preset threshold range, it is judged as abnormal.

As a further implementation of the above technical solution:

The tag ID of step 1) is specifically a natural number other than 0.

As a further implementation of the above technical solution:

The component types on the secondary warehouse panel in step 6) mainly include LED indicator lights, pressure plate switches, and LED digital meters, which are named LED, SWITCH, and METER respectively; the divided areas in step 6) are divided by regional rectangular frames, and each high-voltage switch cabinet secondary warehouse panel image is divided into up to 3 areas, and the areas are named "*P-PSx" in sequence, where "*P" represents the abbreviation of the high-voltage switch cabinet, "PSx" represents the abbreviation of the area, and "x" takes the values of 1, 2, and 3, and the values are not repeated; the regional rectangular frame requires that all the components to be identified be framed according to the component type, and the regional name is used as the key value, and the pixel coordinates of the upper left and lower right corners of the regional rectangular frame are used as the value, and are stored in the Redis database; the cutting regional image in step 6) is to cut the picture according to the regional rectangular frame, and after cutting, the regional image is named "*P-PSx-time.jpg" according to the regional name, "time" is the current number of milliseconds on the computer, and the regional image is saved in the local server; the local server has an operating system of Linux Ubuntu.

As a further implementation of the above technical solution:

In step 7), the marked components are marked with component rectangular frames, each component is named, and the standard state of each component under normal operating conditions of the high-voltage switch cabinet is set. The coordinates of the corner points of the component marking frame and the component name are stored in the database. The component naming is combined with the area name, component type and number. The specific naming rules of the three components are "*P-PSx-LEDy", "*P-PSx-SWITCHy", and "*P-PSx-METERy". "y" represents a non-zero natural number. In a single area image, y has a unique value. The coordinates of the corner points of the component marking frame and the component name are stored, with the component name as the key value, and the component marking frame and the component standard state in the area image as the value, which are stored in the Redis database; in step 7), the cut component image is named for each cut component image, and the component image is named "*P-PSx-LEDy-time.jpg" or "*P-PSx-SWITCHy-time.jpg" or "*P-PSx-METERy-time.jpg" according to the component name, and the component image is saved in the local server.

As a further implementation of the above technical solution:

The calculation of the affine transformation matrix in step 9) is specifically:

Among them, M _2×3 is the affine transformation matrix, which means that the pixel homogeneous coordinates (x ₀ ,y ₀ ,1) in the template image are transformed to the pixel homogeneous coordinates (x ₁ ,y ₁ ,1) at the corresponding position in the non-template image through linear transformation. M can be solved by the corresponding three pairs of pixel coordinates in the template image and the non-template image;

Three pairs of pixel coordinates are selected as the upper left, upper right, and lower right corner pixel coordinates of the AprilTag label. Substituting them into the above formula can solve the affine transformation matrix.

As a further implementation of the above technical solution:

The non-template image component positioning in step 10) is specifically as follows:

A1: Read all data with the key "*P" from the Redis database, traverse and compare the tag ID value in the value with the tag ID value detected in the current non-template image, and when the IDs are equal, obtain the name of the high-voltage switchgear corresponding to the current non-template image;

A2: Read all data with the key "*P-PSx" from the Redis database, where "*P" is the name of the high-voltage switchgear corresponding to the current non-template image. Substitute the pixel coordinates of the upper left and lower right corners of the template image's area rectangle into the following formula to obtain the coordinates of the non-template image's area rectangle. Cut the non-template image according to the transformed area rectangle.

A3: Read all data with the key "*P-PSx-LEDy", "*P-PSx-SWITCHy", and "*P-PSx-METERy" from the Redis database, substitute the pixel coordinates of the upper left and lower right corners of the template image component rectangle into the above formula to obtain the coordinates of the component rectangle of the non-template image, and locate the component of the non-template image area image according to the transformed component rectangle.

As a further implementation of the above technical solution:

Step 14) produces a binary template image of LED numbers 0 to 9, specifically:

B1: Collect multiple digital table component images containing digits 0 to 9, perform image grayscale processing and binarization, and set the binarization threshold to 150, that is, if the grayscale value of the pixel is greater than 150, it is set to 255, otherwise it is set to 0;

B2: Use the cv2.findContours method in OpenCV to find the contour of the binary image and obtain the contour points [(x ₂ ,y ₂ ),...,(x _n ,y _n )]. Then, traverse and compare the maximum and minimum values of x ₁ to x _n and y ₁ to _yn to obtain the upper left corner pixel coordinates (x _min, y _min ) and lower right corner pixel coordinates (x _max ,y _max ) of the minimum rectangle surrounding the digital LED.

B3: Cut the digital table component image according to the four parameters (x, y, w, h), the parameter values are:

B4: Reset the image size of the digital binary template image to (42,51), save the digital template image, and name it "num.jpg" according to the actual meaning of the number. The value range of "num" is a natural number from 0 to 9.

As a further implementation of the above technical solution:

Step 14) calculates the pixel matching degree, reads the single digital binary template image in sequence from numbers 0 to 9 according to the picture name, and calculates the matching degree between the current single digital binary image and the template image:

Among them, _b0 is the binary pixel matrix of the current single digital image, and _b1 is the binary pixel matrix of the read-in template image.

As a further implementation of the above technical solution:

The matching degree sorting in step 14) adopts the bubble sorting method to obtain the maximum matching degree. The number corresponding to the binary template image is the reading result of the current single number and is inserted into the result list.

As a further implementation of the above technical solution:

Step 14) calculates the actual reading of the digital meter, and calculates the reading according to the meaning of the digit size:

Among them, l is the length of the result list, that is, the number of detected numbers, and result(k) is the kth element of the result list.

Working principle:

The images of the secondary compartment panels of the high-voltage switch cabinets were collected through drone inspections. There were 12 high-voltage switch cabinets in total. In a single inspection task, the drone collected 1 image for each high-voltage switch cabinet. A total of 10 inspection tasks were performed at different times, that is, 10 images were collected for each high-voltage switch cabinet to build a sample library. The pixel of a single image is 3000×4000, and 1 template image was selected for each high-voltage switch cabinet.

There are large deviations between the images collected in different batches. The pixel deviations between the remaining 9 images and the template image are statistically analyzed, and the average deviation reaches 132.46 pixels. Therefore, component positioning is required.

According to the type and distribution characteristics of high-voltage switchgear components, the secondary compartment panel images of typical 4P and 6P high-voltage switchgear are divided into regions. All 12 template images are divided into regions, as shown in Table 1.

Table 1 Full name and area division of 12 high-voltage switchgear

Divide the areas according to Table 1, cut the area of the 14P high-voltage switch cabinet secondary compartment panel template image, and cut out the 14P-PS1 indicator light area, 14P-PS2 indicator light area, and 14P-PS3 pressure plate switch area.

Component annotation is performed on the regional images 14P-PS1, 14P-PS2, and 14P-PS3 of the 14P template image to generate component types, component annotation frames, and component standard states.

Read the non-template image of the 14P high-voltage switchgear, detect the label corner point C1 in the non-template image, and read the label corner point C0 in the template image stored in the Redis database:

Select the pixel coordinates of the first three diagonal points and calculate the affine transformation matrix M:

The regional rectangular frame in the 14P template image is mapped to the 14P non-template image through affine transformation, and the regional image is cut.

The component rectangular frame in the regional image of the 14P template image is mapped to the regional image of the 14P non-template image through affine transformation to achieve accurate component positioning.

For the remaining high-voltage switchgear secondary compartment panel images in the sample library, the above process is repeated, and the component images are cut out according to the component annotation box after affine transformation to obtain the indicator light data set, pressure plate switch data set, and digital table data set.

After data augmentation processing such as rotation and flipping, the number of each type of data in the indicator light and pressure plate switch dataset is obtained, as shown in Table 2, and each category of component images in the dataset is divided into training set, verification set, and test set.

Table 2 Indicator light and pressure plate switch data set

The training of the LED indicator and pressure plate switch classification model based on ResNet50 is carried out in the algorithm server. The server operating system is 64-bit Ubuntu16.04. The basic configuration is shown in Table 3. The classification model is implemented based on the Pytorch1.6.0 deep learning framework, and the programming language used is Python2.7.

Table 3 Algorithm server configuration parameters

During the ResNet50 model training process, the ResNet50 model pre-trained on ImageNet is first used to initialize the network weights, and the main training parameters of the network are set as shown in Table 4.

Table 4 Network training parameters

Parameter name Parameter Value Epoch 120 Initial learning rate/Learningrate 1.0 Weight decay factor/Weightdecay 0.0001 Momentum 0.9 Batch Size 32

The training loss and validation loss curves generated by network training at the end of training. As can be seen from the figure, the training loss and validation loss of the network converge synchronously, and the training effect of the ResNet50 classification model is good.

Using the trained classification algorithm model, the component images in the test set were classified and tested. The test results are shown in Table 4-8. The average recognition accuracy reached 99%, which has a very good classification effect.

Table 4-8 Algorithm test results

For the digital table data set, there are 10 in total. According to the method proposed in the present invention, single digital template images of 0 to 9 are produced. Then, the digital table component images are automatically read through grayscale processing, binarization, contour search and other steps. Tested on the digital table data set, the automatic reading accuracy of the digital table reaches 100%.

It can be seen that the present invention can realize the automatic recognition of the secondary compartment panel image of the substation high-voltage switch cabinet with high accuracy, and has the advantages of extremely small amount of image collection data and high versatility, and can be applied to the intelligent inspection system of the substation.

Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A secondary bin panel image recognition system of a transformer substation high-voltage switch cabinet comprises the following steps:

1) Pasting an AprilTag label with unique ID on the horizontal central axis of a secondary bin panel of each high-voltage switch cabinet, and recording a list of the names and the label IDs of the high-voltage switch cabinets in one-to-one correspondence;

2) Collecting sample pictures of the high-voltage switch cabinet to form a sample library, wherein each sample picture comprises a secondary bin panel and an april tag of the high-voltage switch cabinet of the transformer substation;

3) 1 clear picture is selected from a sample library for each high-voltage switch cabinet to serve as a template picture;

4) Invoking an april tag algorithm to detect all tag IDs and tag four corner pixel coordinates in a template picture, calculating the difference value between the tag center point horizontal pixel coordinates and the sample picture horizontal central axis pixel coordinates, and obtaining the tag corresponding to the current high-voltage switch cabinet as the smallest difference value;

5) Searching the name and ID list in the step 1) according to the corresponding tag ID obtained in the step 4), obtaining the name of the high-voltage switch cabinet acquired by the template picture, and storing the name of the template picture, the tag ID and four corner pixel coordinates into a database;

6) Dividing each template map into an indicator light area, a pressing plate switch area and a digital table area according to the types and distribution characteristics of the components on the secondary bin panel of the high-voltage switch cabinet, cutting the images of the areas and naming the images;

7) Marking parts in the region image, naming the names of the parts, cutting the part images one by one according to the part marking frames, and setting the standard state of each part;

8) Sequentially reading non-template pictures from a sample library, and repeating the step 4) to obtain a tag ID and four corner pixel coordinates in each non-template picture;

9) Calculating an affine transformation matrix between the template picture and the non-template picture according to the coordinates of the marked corner pixels in the step 8) and the coordinates of the marked corner pixels in the template picture in the step 4);

10 Mapping the region labeling frame in the template picture into the non-template picture according to the affine transformation matrix, cutting the region image, naming and storing, and mapping the part labeling frame in the template picture into the non-template picture to realize the positioning of the non-template picture component;

11 Cutting part images of each non-template picture in the sample library according to the positioned part labeling frame, and naming each part image to obtain a part image data set;

12 Dividing the data set into an indicator lamp data set, a pressing plate switch data set and a digital table data set according to the names of the component images;

13 Based on the pilot lamp and the pressing plate switch data set obtained in the step 12), carrying out data augmentation, dividing the data set into a training set, a verification set and a test set, importing a data set and a ResNet50 classification model on an algorithm server, training the pilot lamp and the pressing plate switch classification algorithm model based on the ResNet50, and obtaining a pilot lamp on state or an off state and a pressing plate switch on state or an on state;

14 Based on the digital table data set obtained in the step 12), an automatic reading algorithm based on template matching is designed, LED digital binarization template diagrams of 0-9 are manufactured, pixel matching degrees of single digital binarization images of the LED digital table and the digital binarization template diagrams in the new pictures are calculated one by one, single digital reading with the highest matching degree is obtained by sequencing the matching degrees, the digital positions are sequenced according to horizontal coordinates, and actual reading of the digital table is calculated according to the number of bits;

15 Inputting the newly acquired high-voltage switch cabinet picture into a high-voltage switch cabinet image recognition system, repeating the step 4) and the step 5), acquiring the coordinates of four corner pixels of the label in the picture, and repeating the step 6) to acquire an area image;

16 Calculating affine transformation matrixes of the newly acquired picture and the template picture, mapping all component labeling frames in the template picture onto the newly acquired picture, cutting and naming component images, and calling a classification algorithm model based on ResNet50 or an automatic reading algorithm based on template matching according to the component types in the component image names to acquire an identification result;

17 Reading the preset standard state of each indicator light, the pressure plate switch and the digital table, logically judging whether the state of the component is abnormal or not, and giving an alarm in time.

2. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the tag ID of step 1) is specifically a natural number other than 0.

3. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the types of the components on the secondary bin panel in the step 6) mainly comprise LED indicator lamps, a pressing plate switch and an LED digital meter, which are named LED, SWITCH, METER in sequence; the dividing area of the step 6) is divided by adopting an area rectangular frame, the maximum of 3 areas are divided by the secondary bin panel image of each high-voltage switch cabinet, the areas are sequentially named as 'P-PSx', wherein 'P' represents the short name of the high-voltage switch cabinet, 'PSx' represents the short name of the area, and 'x' takes values of 1, 2 and 3 and the values are not repeated; the regional rectangular frame is required to be framed to all the parts to be identified according to the type of the parts, and pixel coordinates of upper left corner points and lower right corner points of the regional rectangular frame are value values according to the regional name as key values and are stored in a Redis database; the step 6) of cutting the area image, cutting the picture according to the area rectangular frame, naming the area image as 'P-PSx-time. Jpg' according to the area name after cutting, wherein 'time' is the current millisecond number of the computer, and storing the area image in a local server; and the local server is provided with an operating system of Linux Ubuntu.

4. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the labeling component in the step 7) adopts component rectangular frame labeling, names each component, sets the standard state of each component in the normal operation state of the high-voltage switch cabinet, stores the corner coordinates of the component labeling frame and the component names in a database, and stores the component names, the component standard states of the component labeling frame and the component in the region image as value in terms of key values by combining the region names, the component types and the numbers, wherein the specific naming rules of the three components are "xP-PSx-LEDy", "x-P-PSx-SWITCHy", "x-P-PSx-METERy", "y" represent nonzero natural numbers; the cutting part image of the step 7) is named after each part image is cut, the part image is named as 'P-PSx-LEDy-time. Jpg' or 'P-PSx-SWITCHy-time. Jpg' or 'P-PSx-METERy-time. Jpg' according to the part name, and the part image is stored in a local server.

5. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the calculated affine transformation matrix in the step 9) is specifically:

wherein M is _2×3 For affine transformation matrix, representing the pixel homogeneous coordinates (x ₀ ,y ₀ 1) transforming to corresponding position pixel homogeneous coordinates (x) ₁ ,y ₁ 1), obtaining M through solving corresponding three pairs of pixel coordinates in a template picture and a non-template picture;

and selecting three pairs of pixel coordinates, namely, the pixel coordinates of the upper left corner, the upper right corner and the lower right corner of the april tag label, and substituting the pixel coordinates into the pixel coordinates to solve the affine transformation matrix.

6. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the positioning of the non-template diagram component in the step 10) is specifically as follows:

a1: reading data with all keys of P from a Redis database, traversing and comparing a tag ID value in a value with a tag ID value detected in a current non-template picture, and obtaining a high-voltage switch cabinet name corresponding to the current non-template picture when the IDs are equal;

a2: reading data with all keys of P-PSx from a Redis database, wherein P is the name of a high-voltage switch cabinet corresponding to the current non-template picture, substituting pixel coordinates of upper left corner points and lower right corner points of a template picture region rectangular frame into the following formula to obtain region rectangular frame coordinates of the non-template picture, and cutting the non-template picture according to the transformed region rectangular frame;

a3: reading data of which all keys are "+" -P-PSx-LEDy "," -P-PSx-SWITCHy "," -P-PSx-METERy ", substituting the pixel coordinates of the upper left corner and the lower right corner of the rectangular frame of the template picture component into the upper formula, and obtaining the coordinates of the rectangular frame of the component of the non-template picture, and positioning the component of the non-template picture region image according to the transformed rectangular frame of the component.

7. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the step 14) of manufacturing the LED digital binarization template chart with 0 to 9 comprises the following specific steps:

b1: collecting a plurality of digital table part images containing 0-9 digits, performing image gray scale processing and binarization processing, wherein a binarization threshold value is set to 150, namely, if the gray scale value of a pixel point is greater than 150, 255 is set, otherwise, 0 is set;

b2: searching the outline of the binary image by using a cv2.findContours method in OpenCV to obtain outline points [ (x) ₂ ,y ₂ ),...,(x _n ,y _n )]Traversal comparison x ₁ ～x _n And y ₁ ～y _n And the minimum and maximum of (2) to obtain the upper left corner pixel coordinate (x) _min, y _min ) And lower right corner pixel coordinates (x _max ,y _max )；

B3: cutting the digital table part image according to four parameters of (x, y, w and h), wherein the parameters are as follows:

b4: the image size of the digital binarization template image is reset and uniformly set as (42, 51), the digital template image is saved, and the numerical template image is named as 'num. Jpg' according to the actual meaning of the digital, and the value range of 'num' is natural numbers 0-9.

8. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the step 14) of calculating the pixel matching degree, sequentially reading single digital binarization template images from the numbers 0 to 9 according to the picture names, and calculating the matching degree of the current single digital binarization image and the template image:

wherein b ₀ Binarized pixel matrix for current single digital image, b ₁ The pixel matrix is binarized for the read template map.

9. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: and 14) sorting the matching degree, namely obtaining the maximum value of the matching degree by adopting an bubbling sorting method, wherein the number corresponding to the binarization template diagram is the reading result of the current single number, and inserting the reading result into a result list.

10. The substation high-voltage switch cabinet secondary bin panel image recognition system according to claim 1, wherein: the actual reading of the digital table is calculated in the step 14), and the reading is calculated according to the meaning of the number of bits:

where l is the length of the result list, i.e. the number of digits detected, and result (k) is the kth element of the result list.

Images