CN114332051A - Image-inference-based power transmission and distribution line equipment asset general survey method - Google Patents

Abstract

The invention discloses a power transmission and distribution line equipment asset general survey method based on image reasoning, which fuses data, a model, report generation and the like and can be conveniently deployed in an enterprise for use. The method comprises the following steps: s1, finely acquiring a tower body image, an insulation sub-image, a ground wire part image, a hardware part image, an online detection device and a lightning arrester image by using an unmanned aerial vehicle, marking information required by the acquired images, and constructing a data set; s2, constructing a YOLOv5 model, setting training parameters, and training by using a training set; s3, testing the trained model by using the test set, and evaluating the model by using the recall rate and the accuracy rate; s4, adjusting the data or model parameters of the training set according to the evaluation result of S3, and continuing iterative training until an ideal effect is achieved; and S5, detecting the images acquired by the unmanned aerial vehicles of a certain power transmission and distribution line based on the optimal model, and performing logic post-processing according to the detection result to obtain the number of each equipment body.

Description

Image-inference-based power transmission and distribution line equipment asset general survey method

Technical Field

The invention relates to the technical field of artificial intelligence such as computer vision, deep learning and the like, in particular to a power transmission and distribution line equipment asset general survey method based on image reasoning.

Background

Compared with the automatic asset census of the patent, the manual asset census method has higher time cost, labor cost and safety risk and is not beneficial to high-efficiency asset clearing; the mode of constructing the power supply line three-dimensional model through unmanned aerial vehicle oblique photography for asset census also needs higher time and technical cost, and the technology is more suitable for three-dimensional modeling of the overall terrain of the power transmission line instead of equipment clearing acting on a single power supply tower.

Compared with a deep learning algorithm used in the patent, the digital image processing method of unmanned aerial vehicle line patrol and traditional machine learning identification is not flexible enough, the feature extractor needs to be designed elaborately by an algorithm engineer, different feature extractors need to be designed for detecting different objects, and target detection is performed by using a digital image processing method, so that the accuracy rate is often inferior to that of the deep learning method.

The deep learning algorithm has a higher pursuit on detection speed and detection accuracy, wherein two steps are required for a two-stage detection target, a candidate region is generated in an image firstly, then the candidate region is input into a full-connection layer again for classification and regression, and each candidate frame region needs to be subjected to the classification and regression, so that the time is very long. The one-stage algorithm directly generates the class probability and the position coordinates of the object without a stage of generating a candidate region, thereby greatly saving the operation time. Common one-stage algorithms include SSD, YOLO series, etc., in which the YOLOv5 algorithm has excellent performance in both detection speed and detection accuracy.

Disclosure of Invention

The invention designs a method for rapidly detecting image sample annotation result data, which is used for detecting equipment on a power transmission and distribution line and generating a complete equipment census document. Compared with the traditional manual census and the traditional digital image processing method, the accuracy of detection and the accuracy of equipment census are greatly improved.

The invention provides a power transmission and distribution line equipment asset general survey method based on image reasoning, which comprises the following steps:

s1, the unmanned aerial vehicle finely acquires the tower body image, the insulation sub-image, the ground wire part image, the hardware part image, the online detection device and the lightning arrester image, marks the information required by the acquired image,

constructing a data set;

s2, constructing a YOLOv5 model, setting training parameters, and training by using a training set;

s3, testing the trained model by using the test set, and evaluating the model by using the recall rate and the accuracy rate;

s4, adjusting the data or model parameters of the training set according to the evaluation result of S3, and continuing iterative training until an ideal effect is achieved;

and S5, detecting the images acquired by the unmanned aerial vehicles of a certain power transmission and distribution line based on the optimal model, and performing logic post-processing according to the detection result to obtain the number of each equipment body.

Alternatively to this, the first and second parts may,

step S1 includes data collection, cleaning and labeling, and requires strict formulation of labeling specification, category definition, and the like.

Alternatively to this, the first and second parts may,

steps S3 and S4 are model iteration processes, and various hyper-parameters need to be adjusted continuously to make the detection result reach a better degree, so as to achieve the purpose of accurately counting the number of appliances of each device.

Alternatively to this, the first and second parts may,

step S5 is to perform quantity statistics according to the detected devices and output census statistical results.

Alternatively to this, the first and second parts may,

the YOLOv5 model is mainly composed of four parts: the device comprises an input end, a backhaul reference network, a neutral middle layer and a Head output layer;

the input end is used for preprocessing input data and enhancing the data;

the Neck intermediate layer is used for further extracting features from a BackBone reference network, outputting three tensors with different scales according to the size of an object to be identified, and respectively identifying a small target, a medium target and a large target;

the Head output layer is used for outputting the bounding box identifying the object and the corresponding category for three different sizes.

Alternatively to this, the first and second parts may,

YOLOv5 measures the difference between the detected frame and the real frame using GIOU _ loss, which overcomes the problem that the loss function of IOU _ loss cannot be optimized when the detected frame and the object do not intersect, and has scale invariance compared with the traditional regression loss function. The calculation formula of GIOU _ loss is as follows:

wherein a is a detection frame and b is a real frame. The IOU is an intersection-union ratio, namely the ratio of the intersection and union of the detection frame and the real frame, and C is a minimum circumscribed matrix capable of enclosing a and b.

Alternatively to this, the first and second parts may,

the data set in step S1 is uploaded to the server side through the client side, and a request for a desired service is sent.

Alternatively to this, the first and second parts may,

the server in step S2 calls an existing model to detect data according to the uploaded data and the service request, and returns a result to the client after obtaining the model detection data and completing post-processing.

Alternatively to this, the first and second parts may,

and the user generates an asset census report after the client obtains the return result.

Alternatively to this, the first and second parts may,

and the server counts and calculates the number of each equipment according to the detection result and returns the counting result to the client.

Compared with the prior art, the application has the following beneficial effects:

the invention integrates data, models, report generation and the like into a set of integrated system, the client is responsible for instruction command issue, the server is used for detecting results and providing final results, and the system standardizes the flow of the general survey method and can be conveniently deployed in enterprises for use. The method is based on the detection result of the YOLOV5 model, strict logic judgment processing is carried out, and a census report is finally generated, so that the electric tower census personnel can conveniently examine the assets subsequently, and the time cost and the safety risk of the working personnel are greatly reduced. By adopting the advanced one-stage algorithm YOLOv5, the problems of single characteristic and difficult design of the traditional digital image processing are solved, the type of model detection can be conveniently expanded, and the method has high expandability and high accuracy.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow chart of an embodiment of the method for census of assets of power transmission and distribution line equipment based on image inference;

FIG. 2 is a system deployment scenario of an embodiment of the image-inference-based asset census method for power transmission and distribution lines;

fig. 3 is a schematic diagram of the principle of an embodiment of the power transmission and distribution line equipment asset census method based on image inference.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The words "a", "an" and "the" and the like as used herein are also intended to include the meanings of "a plurality" and "the" unless the context clearly dictates otherwise. Furthermore, the terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

The invention provides a method, a system and a device based on image inference, which firstly introduces the image inference method based on a Yolov5 model and secondly introduces the system and the device based on the model. The target detection algorithm uses a one-stage detection algorithm YOLOv5 which is excellent at both academic and industrial stages, and the realization and the use of the target detection algorithm are quite mature and convenient, so that the accuracy and the recall rate of the target detection are improved, the time for model development is saved, and sufficient time is reserved for engineers in subsequent important logic processing.

The present embodiment is described below with reference to fig. 1 to 3:

the method for identifying the power supply tower equipment by using the YOLOv5 model comprises the following steps:

and S1, finely acquiring the tower body image, the insulation sub-image, the ground wire part image, the hardware part image, the online detection device and the lightning arrester image by using the unmanned aerial vehicle, marking the information required by the acquired image, and constructing a data set.

And S2, constructing a YOLOv5 model, setting training parameters, and training by using a training set.

And S3, testing the trained model by using the test set, and evaluating the model by using the recall rate and the accuracy rate.

And S4, adjusting the data or model parameters of the training set according to the evaluation result of S3, and continuing iterative training until the ideal effect is achieved.

And S5, detecting the images acquired by the unmanned aerial vehicles of a certain power transmission and distribution line based on the optimal model, and performing logic post-processing according to the detection result to obtain the number of each equipment body.

The step S1 is mainly for data collection, cleaning and labeling, and requires strict formulation of labeling specifications, category definitions, and the like. The step S2 requires training. S3 and S4 are model iteration processes, and various hyper-parameters need to be adjusted continuously to enable the detection result to reach a better degree, so that the purpose of accurately counting the number of appliances of each device is achieved.

And S5 is a final post-processing step, and the general survey statistical result is output according to the number statistics of the detected equipment.

The YOLOv5 model is mainly composed of four parts: the device comprises an input end, a backhaul reference network, a neutral middle layer and a Head output layer.

Input of the model: and preprocessing and enhancing the input data. Preprocessing involves scaling the input images to meet the required size of the network, and normalizing the images to improve the speed of training and the accuracy of the network. The data enhancement comprises the steps of randomly erasing an input image, translating, rotating, mirroring, randomly increasing noise, and expanding data by using Mix-up, Mosaic and SAT, wherein the Mosaic data enhancement utilizes four pictures to splice the four pictures, each picture has a corresponding label, a new picture is obtained after the four pictures are spliced, the labels corresponding to the pictures are also obtained, the background of the detected object is enriched while the detected object is relatively reduced, and the detection capability of the model for small targets is effectively improved.

The BackBone reference network is characterized in that an input image is firstly subjected to downsampling without information loss through a Facus module, data characteristics are extracted through CSPDarknet, the CSPDarknet is a deep network structure and has enough receptive field, a CSP module is added on the basis of Darknet to solve the problem of repeated gradient information during large-scale network optimization, the CSP divides the characteristic mapping of a basic layer into two parts for processing, so that gradient flow is transmitted through different network paths, repeated calculation of gradient is reduced, and then the gradient flow and the gradient flow are combined through a cross-layer splicing structure, and the calculated amount is reduced while the model accuracy is ensured. And a smooth Mish activation function is used, so that the hard zero boundary of the ReLU is avoided, more information is allowed to enter the neural network, and the accuracy and generalization capability of the model are improved. Dropblock is used in the model in an inserting mode, characteristics of adjacent regions are discarded, the defect that Dropout has an unobvious effect on the convolutional layer is overcome, overfitting is prevented beneficially, and generalization capability of the model is improved.

And (3) a neutral layer: and further extracting features from the BackBone, and outputting three tensors with different scales according to the size of the object to be identified, wherein the three tensors are respectively used for identifying a small target, a medium target and a large target. And a CSP2 module, an SPP module and an FPN + PAN structure are added to further fuse and extract diversity characteristics. CSP2 has borrowed the CSP module, has changed the residual error structure in the CSP into convolution structure, CSP2 has further strengthened the ability that the network fuses to the characteristic. The SPP (spatial Pyramid Pooling networks) performs maximum Pooling by using Pooling cores with the sizes of 1, 5, 9 and 13, and then the results are spliced together to perform multi-scale fusion on the features. The FPN (feature Pyramid networks) feature Pyramid network enlarges the original feature map layer by layer from top to bottom through deconvolution to construct a feature Pyramid, which is beneficial to semantic features of objects with different sizes extracted by the network, so that the model can identify the same object with different sizes and scales. The PAN (Path Aggregation network) draws features from bottom to top by using a PANET algorithm in the field of image segmentation, and the parameters of different layers from the FPN are aggregated by a splicing method, so that the positioning capability of the network is enhanced.

The Head output layer outputs a bounding box and a corresponding category of an identified object aiming at three different sizes, YOLOv5 measures the difference between a detection box and a real box by using GIOU _ loss, and the GIOU _ loss overcomes the problem that a loss function cannot be optimized when the detection box and the object do not intersect, and has scale invariance compared with a traditional regression loss function. The calculation formula of GIOU _ loss is as follows:

wherein a is a detection frame and b is a real frame. The IOU is an intersection-union ratio, namely the ratio of the intersection and union of the detection frame and the real frame, and C is a minimum circumscribed matrix capable of enclosing a and b.

Using the YOLOv5 network for object recognition of images, each image would yield 22743 preselected boxes, each containing 5+ n values, the center point coordinates x, y and width height w, h of the preselected box, and the confidence c and probability of n classes. Firstly, filtering some preselected frames with low confidence degrees according to the value of the confidence degree c, wherein the confidence degree threshold value is 0.5 in the model, carrying out GIOU-NMS screening on the retained preselected frames to screen out excessive repeated preselected frames, and taking the remaining preselected frames as final detection results.

Next, the main components of the system of the present invention include a hardware layer, a network layer, an AI platform layer, and a service layer. The system architecture diagram is as follows

The system deployment scenario is as follows:

unmanned aerial vehicle patrols and examines through becoming more meticulous and obtains data, and the user uploads data to the server side through the client, sends required service request simultaneously. And the server calls the existing model to detect the data according to the uploaded data and the service request, and returns a result to the client after obtaining the model detection data and finishing post-processing. After the user obtains the return result at the client, the user can automatically generate the asset census report by one key.

The main service flow chart of the system is as follows:

and uploading the fine inspection image data of the unmanned aerial vehicle to a server through a client, and clicking by a user to send a service request. And the server stores the uploaded image data, calls the existing model to detect the image data after receiving the command, and stores the detection. And the server counts and calculates the number of the equipment and appliances according to the detection result, and returns the counting result to the client. And the client uses the returned result data and calls a report template to finally generate the asset census report.

In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.

A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Artificial intelligence is the subject of studying computers to simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural voice processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. A power transmission and distribution line equipment asset census method based on image reasoning is characterized by comprising the following steps:

s1, finely acquiring a tower body image, an insulation sub-image, a ground wire part image, a hardware part image, an online detection device and a lightning arrester image by using an unmanned aerial vehicle, marking information required by the acquired images, and constructing a data set;

s2, constructing a YOLOv5 model, setting training parameters, and training by using a training set;

s3, testing the trained model by using the test set, and evaluating the model by using the recall rate and the accuracy rate;

s4, adjusting the data or model parameters of the training set according to the evaluation result of S3, and continuing iterative training until an ideal effect is achieved;

and S5, detecting the images acquired by the unmanned aerial vehicles of a certain power transmission and distribution line based on the optimal model, and performing logic post-processing according to the detection result to obtain the number of each equipment body.

2. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

step S1 includes data collection, cleaning and labeling, and requires strict formulation of labeling specification and category definition.

3. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

steps S3 and S4 are model iteration processes, and various hyper-parameters need to be adjusted continuously to make the detection result reach a better degree, so as to achieve the purpose of accurately counting the number of appliances of each device.

4. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

step S5 is to perform quantity statistics according to the detected devices and output census statistical results.

5. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

the YOLOv5 model is mainly composed of four parts: the device comprises an input end, a backhaul reference network, a neutral middle layer and a Head output layer;

the input end is used for preprocessing input data and enhancing the data;

the Neck intermediate layer is used for further extracting features from a BackBone reference network, outputting three tensors with different scales according to the size of an object to be identified, and respectively identifying a small target, a medium target and a large target;

the Head output layer is used for outputting the bounding box identifying the object and the corresponding category for three different sizes.

6. The image inference-based power transmission and distribution line equipment asset census method of claim 5, wherein:

YOLOv5 measures the difference between the detected frame and the real frame using GIOU _ loss, which overcomes the problem that the loss function of IOU _ loss cannot be optimized when the detected frame and the object do not intersect, and has scale invariance compared with the traditional regression loss function. The calculation formula of GIOU _ loss is as follows:

wherein a is a detection frame and b is a real frame. The IOU is an intersection-union ratio, namely the ratio of the intersection and union of the detection frame and the real frame, and C is a minimum circumscribed matrix capable of enclosing a and b.

7. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

the data set in step S1 is uploaded to the server side through the client side, and a request for a desired service is sent.

8. The image inference-based power transmission and distribution line equipment asset census method of claim 1, wherein:

the server in step S2 calls an existing model to detect data according to the uploaded data and the service request, and returns a result to the client after obtaining the model detection data and completing post-processing.

9. The image inference-based power transmission and distribution line equipment asset census method of claim 8, wherein:

and the user generates an asset census report after the client obtains the return result.

10. The image inference-based power transmission and distribution line equipment asset census method of claim 8, wherein:

and the server counts and calculates the number of each equipment according to the detection result and returns the counting result to the client.

Images