CN114035606A - Pole tower inspection system, pole tower inspection method, control device and storage medium - Google Patents
Pole tower inspection system, pole tower inspection method, control device and storage medium Download PDFInfo
- Publication number
- CN114035606A CN114035606A CN202111308904.7A CN202111308904A CN114035606A CN 114035606 A CN114035606 A CN 114035606A CN 202111308904 A CN202111308904 A CN 202111308904A CN 114035606 A CN114035606 A CN 114035606A
- Authority
- CN
- China
- Prior art keywords
- tower
- abnormal
- image
- model
- state information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 77
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000002159 abnormal effect Effects 0.000 claims abstract description 155
- 238000001514 detection method Methods 0.000 claims abstract description 45
- 238000004590 computer program Methods 0.000 claims description 6
- 230000005856 abnormality Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 description 14
- 238000012360 testing method Methods 0.000 description 12
- 238000011161 development Methods 0.000 description 10
- 230000018109 developmental process Effects 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 230000003416 augmentation Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 239000012212 insulator Substances 0.000 description 4
- 241000264877 Hippospongia communis Species 0.000 description 3
- 235000005770 birds nest Nutrition 0.000 description 3
- 238000004880 explosion Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 235000005765 wild carrot Nutrition 0.000 description 3
- 244000000626 Daucus carota Species 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Abstract
The application relates to a pole tower inspection system, a pole tower inspection method, a control device and a storage medium. The pole tower inspection system comprises: a flying device; the acquisition device is arranged on the flying device and used for acquiring the tower image; and the control device is arranged on the flying device, is connected with the acquisition device, is used for acquiring tower images, is also used for determining the abnormal state information of the tower according to the tower images and the tower abnormal model, and is also used for sending the abnormal state information. And the terminal equipment is connected with the control device and used for receiving the abnormal state information sent by the control device. The tower inspection system determines the abnormal state information of the tower at the acquisition end of the tower image, and the acquired image does not need to be sent to a remote server for judgment, so that the detection efficiency is improved.
Description
Technical Field
The application relates to the technical field of tower detection, in particular to a tower inspection system, a tower inspection method, a control device and a storage medium.
Background
Along with the continuous development and construction of the power grid, the inspection workload aiming at the power facilities is also continuously increased, the inspection difficulty is further increased particularly in some remote areas, and meanwhile, the management mode is continuously upgraded by a power grid management unit.
At present, a fixed camera online detection system is adopted for pole tower inspection, a camera arranged on a pole tower is used for acquiring a video or an image of the fixed camera online detection system, the video or the image is transmitted to a remote server, and the server detects and identifies an abnormality of a target. However, the effect of network signals on transmitting pictures to the remote server is large, especially when the transmitted images are high-definition images, signals are intermittent in remote areas, and data are difficult to transmit in real time, so that the detection efficiency of the tower inspection by adopting the fixed camera online detection system is low.
Disclosure of Invention
In view of the above, it is necessary to provide a tower inspection system, a tower inspection method, a control device, and a storage medium with high detection efficiency.
In a first aspect, a pole tower inspection system is provided, including: a flying device; the acquisition device is arranged on the flying device and used for acquiring the tower image; and the control device is arranged on the flying device, is connected with the acquisition device, is used for acquiring tower images, is also used for determining the abnormal state information of the tower according to the tower images and the tower abnormal model, and is also used for sending the abnormal state information. And the terminal equipment is connected with the control device and used for receiving the abnormal state information sent by the control device.
In one embodiment, the device further comprises an adjusting device for adjusting the acquisition angle of the acquisition device.
In a second aspect, a pole tower inspection method is provided, which includes: acquiring a tower image; determining abnormal state information of the tower according to the tower image and the tower abnormal model; and sending the abnormal state information to terminal equipment.
In one embodiment, the step of determining the abnormal state information of the tower according to the tower image and the tower abnormal model includes: determining the abnormal position of the tower according to the tower image and the first tower abnormal model; controlling an acquisition device to acquire a detail image of the abnormal position of the tower according to the abnormal position of the tower; the acquisition device is used for acquiring images of the tower; and determining the abnormal state information of the tower according to the detail image and the second tower abnormal model.
In one embodiment, the step of controlling the acquisition device to acquire the detailed image of the abnormal position of the tower according to the abnormal position of the tower includes: calculating the pixel deviation between the abnormal position of the tower and the tower image; calculating a target angle required to be adjusted by the acquisition device according to the pixel deviation; and controlling the acquisition device according to the target angle, controlling the acquisition device to focus on the abnormal position of the tower, and acquiring a detail image of the abnormal position of the tower.
In one embodiment, the method further comprises: controlling a flying device to fly according to a preset inspection route; wherein the flying device is provided with the collecting device; and controlling the acquisition device to acquire the tower image under the condition that the flying device flies to a detection point.
In one embodiment, the method further comprises: and if the abnormal position of the tower is determined according to the tower image and the first tower abnormal model, controlling the flying device to fly to the next detection point according to the preset routing inspection route.
In one embodiment, the first tower anomaly model is a YOLOv4 model, and/or the second tower anomaly model is a YOLOv4 model.
In a third aspect, a tower inspection device is provided, comprising: the acquisition module is used for acquiring a tower image; the determining module is used for determining the abnormal state information of the tower according to the tower image and the tower abnormal model; and the sending module is used for sending the abnormal state information to the terminal equipment.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the pole tower inspection method according to any one of the second aspects.
The pole tower inspection system comprises a flying device, a collecting device, a control device and terminal equipment, wherein the control device determines abnormal state information of a pole tower according to pole tower images collected by the collecting device and pole tower abnormal models and sends the abnormal state information to the terminal equipment. The tower inspection system determines the abnormal state information of the tower at the acquisition end of the tower image, and the acquired image does not need to be sent to a remote server for judgment, so that the detection efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the conventional technologies of the present application, the drawings used in the descriptions of the embodiments or the conventional technologies will be briefly introduced below, it is obvious that the drawings in the following descriptions are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a tower inspection system in a first embodiment;
fig. 2 is a schematic flow chart of a tower inspection method in one embodiment;
FIG. 3 is a flowchart illustrating a method for determining abnormal state information according to an embodiment;
FIG. 4 is a flowchart illustrating a method for building the YOLOv4 model according to an embodiment;
FIG. 5 is a flowchart illustrating a method for obtaining a detail image according to an embodiment;
fig. 6 is a schematic structural diagram of a tower inspection system in the second embodiment;
fig. 7 is a schematic structural diagram of a tower inspection system in the third embodiment;
fig. 8 is a block diagram of the tower inspection device in one embodiment.
Detailed Description
To facilitate an understanding of the present application, the present application will now be described more fully with reference to the accompanying drawings. Embodiments of the present application are set forth in the accompanying drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.
Spatial relational terms, such as "under," "below," "under," "over," and the like may be used herein to describe one element or feature's relationship to another element or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements or features described as "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the exemplary terms "under" and "under" can encompass both an orientation of above and below. In addition, the device may also include additional orientations (e.g., rotated 90 degrees or other orientations) and the spatial descriptors used herein interpreted accordingly.
It will be understood that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or be connected to the other element through intervening elements. Further, "connection" in the following embodiments is understood to mean "electrical connection", "communication connection", or the like, if there is a transfer of electrical signals or data between the connected objects.
As used herein, the singular forms "a", "an" and "the" may include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises/comprising," "includes" or "including," etc., specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof. Also, as used in this specification, the term "and/or" includes any and all combinations of the associated listed items.
The pole tower detection indicates that the staff utilizes relevant equipment, discerns power equipment such as insulator, fastener, equalizer ring and affiliated facilities thereof to detect the relevant unusual of pole tower, if: the insulator has the self-explosion phenomenon, bird nest, honeycomb and other foreign matters, and the tower has construction facilities and the like.
Because some devices are relatively small, a high-definition camera is mostly adopted to collect related images, and then the collected images are transmitted to a remote server for processing, although the detection of the tower can be realized, the method has at least the following defects that firstly, network delay exists when data needs to be transmitted back to the remote server, and even network interruption occurs, so that the detection delay even cannot be detected; secondly, a large amount of services are often arranged on the remote server, and a large amount of waiting time delay is generated for data processing in the use peak period; thirdly, when the number of the collecting devices is increased, the network or the server of the machine room needs to be expanded after the load of the machine room is exceeded, and the operation is very inconvenient.
In view of this, the embodiment of the application provides a pole tower inspection system, and this pole tower inspection system includes flight device, collection system, controlling means and terminal equipment. And the control device determines the abnormal state information of the tower according to the tower image acquired by the acquisition device, the tower image and the tower abnormal model, and sends the abnormal state information to the terminal equipment. The tower inspection system determines the abnormal state information of the tower at the acquisition end of the tower image, and the acquired image does not need to be sent to a remote server for judgment, so that the detection efficiency is improved.
Referring to fig. 1, a tower inspection system provided in an embodiment of the present application is shown, and as shown in fig. 1, the tower inspection system may include a flying device 120, a collecting device 140, a control device 160, and a terminal device 180.
The collecting device 140 is disposed on the flying device 120 and is used for collecting tower images. Optionally, the flying device 120 may include a drone, and the collecting device 140 is disposed at the bottom of the drone. Optionally, the collecting device 140 is a camera, and can collect tower images.
The control device 160 is disposed on the flying device 120, connected to the collecting device 140, and configured to acquire tower images, determine abnormal state information of the tower according to the tower images and the tower abnormal model, and send the abnormal state information. The abnormal state information may include, but is not limited to, existence of foreign matters such as bird's nests and honeycombs, self-explosion of insulators, existence of construction machinery under wires, dropping of grading rings, dropping of bolts, and dropping of pins. The tower abnormity model can be used for detecting whether the tower image has an abnormal state. In one embodiment, the control device 160 may further store the abnormal state information correspondingly. Therefore, the staff can determine the condition of the abnormal tower according to the information stored in the control device 160, and the condition that the abnormal state information of the tower is lost due to the interruption of the transmission network is avoided. Optionally, the control device 160 includes a development board, and the development board provides the required calculation power and storage space for determining the abnormal state information of the tower, so as to implement the function of edge calculation. According to the embodiment, the development board is used for performing edge calculation, and the tower image is processed at the tower image acquisition end, so that network time delay caused by image transmission is avoided, and the detection efficiency is improved.
The terminal device 180 is connected to the control device 160, and is configured to receive the abnormal state information sent by the control device 160, and a worker may monitor the tower according to the information received by the terminal device 180. The detection result of the control device 160 is transmitted to the terminal device 180 in real time, so that the detection process is conveniently controlled by the staff. Optionally, the terminal device 180 includes a display screen, and the terminal device 180 displays the received abnormal state information through the display screen. In one embodiment, the control device 160 may also be configured to transmit the tower image collected by the collecting device 140 to the terminal device 180. So that the staff can obtain more comprehensive information. Optionally, the terminal device 180 may include a tablet computer for receiving the return information of the unmanned aerial vehicle, so as to facilitate monitoring and detecting conditions by the staff. In one embodiment, the control device 160 establishes a wireless connection with the terminal device 180, so that the terminal device 180 acquires the abnormal state information of the tower.
The flight device of the tower inspection system provided by the embodiment carries the control device and the acquisition device, the control device is used for realizing edge calculation, the images are processed at the acquisition end of the tower images, time delay caused by the fact that the tower images are transmitted to the remote server can be avoided, and the detection efficiency is improved. Meanwhile, the tower inspection system provided by the embodiment does not need to perform data interaction with a remote server, and can still work normally in areas without networks, so that the availability of the tower inspection system is improved.
Meanwhile, under the condition that more flying devices, acquisition devices and control devices need to be added to the pole tower inspection system provided by the embodiment, the network load and the calculation load of a data center do not need to be considered, the deployment is flexible and convenient, and the expandability is strong. The control device transmits the abnormal state information of the tower back to the terminal equipment in real time, so that the emergency situation can be conveniently handled by the staff, and the working efficiency is improved.
In an embodiment, the pole tower inspection system provided by the above embodiment may further include an adjusting device for adjusting the collecting angle of the collecting device. In one embodiment, the control device is connected with the adjusting device and used for controlling the adjusting device to adjust the collecting angle of the collecting device. The acquisition device is installed on the adjusting device, and the adjusting device can adjust the horizontal and pitching angles of the acquisition device, so that the acquisition of tower images by the acquisition device is realized. Optionally, under the condition that the acquisition device is a camera, the adjusting device may be a cradle head. Optionally, the cradle head is an electric cradle head, the electric cradle head comprises two actuating motors, the motors are connected with the control device, and the motors move according to received control signals of the control device, so that the acquisition angle of the acquisition device is adjusted. Above-mentioned embodiment adjusts adjusting device through controlling means, for receiving remote server's regulation signal, there is not the network time delay problem, avoided because there is the time delay in the network transmission to lead to unmanned aerial vehicle hover time overlength to reduce detection efficiency's phenomenon, still avoided unmanned aerial vehicle position, angle change during hovering simultaneously, the adjustment data that remote server returned is invalid, make the unstable problem of detection effect, and improved and patrolled and examined efficiency.
Optionally, the flying device may include an unmanned aerial vehicle, the adjusting device may include a three-dimensional stability augmentation platform, the collecting device may include a high definition camera, and the control device may include a development board. Specifically, the three-dimensional stability augmentation platform is installed under the unmanned aerial vehicle, and the high definition digtal camera is installed under the three-dimensional stability augmentation platform, and the development board is installed inside the unmanned aerial vehicle, provides the computing power for confirming the abnormal state information of shaft tower and controlling the high definition digtal camera.
Referring to fig. 2, a tower inspection method according to a first embodiment of the present disclosure is shown, and as shown in fig. 2, the tower inspection method may include steps S202 to S206.
And S202, acquiring a tower image.
In one embodiment, the tower inspection system includes a collection device through which tower images are acquired. The description of the collecting device is given in detail in the above embodiments and will not be repeated here.
And S204, determining the abnormal state information of the tower according to the tower image and the tower abnormal model.
The abnormal state information may include, but is not limited to, existence of foreign matters such as bird's nests and honeycombs, self-explosion of insulators, existence of construction machinery under wires, dropping of grading rings, dropping of bolts, and dropping of pins. The tower abnormity model can be used for detecting whether the tower image has an abnormal state. In one embodiment, the acquired tower image is input into a tower abnormity model, and the tower abnormity model outputs the abnormity state information of the tower.
And S206, sending the abnormal state information to the terminal equipment.
In one embodiment, the terminal device displays the abnormal state information, so that a worker can obtain the abnormal state information of the tower and detect the condition of the tower.
According to the pole tower inspection method provided by the embodiment, the pole tower image is obtained, the abnormal state information of the pole tower is determined according to the pole tower image and the pole tower abnormal model, the abnormal state information is sent to the terminal equipment, data interaction with a remote server is not needed by adopting an edge calculation mode, and the detection efficiency is improved.
Referring to fig. 3, a method for determining abnormal state information according to an embodiment of the present application is shown, and as shown in fig. 3, the method for determining abnormal state information may include steps S302 to S306.
S302, determining the abnormal position of the tower according to the tower image and the first tower abnormal model.
It should be noted that the first tower anomaly model may be used to determine an anomaly position of a tower according to a tower image. In one embodiment, the acquired tower image is input into a first tower abnormity model, and the first tower abnormity model outputs the abnormity position of the tower. In one embodiment, the first tower abnormal model may output tower abnormal state information, and determine the tower abnormal position according to the tower abnormal state information.
In one embodiment, if the abnormal position of the tower is determined according to the tower image and the first tower abnormal model, the flying device is controlled to fly to the next detection point according to a preset routing inspection route.
S304, controlling the acquisition device to acquire the detail image of the abnormal position of the tower according to the abnormal position of the tower.
The acquisition device is used for acquiring images of the tower. Optionally, the collecting device may be a camera. The detail image contains more image information of the abnormal position of the tower relative to the tower image. Optionally, the detail image is an enlarged image of an abnormal position of the tower. And the detail image is a local image corresponding to the abnormal position of the tower image.
And S306, determining the abnormal state information of the tower according to the detail image and the second tower abnormal model.
It should be noted that the second tower anomaly model may be used to determine anomaly status information of the tower according to the detail image. In one embodiment, the acquired detail image is input into a second tower abnormity model, and the second tower abnormity model outputs abnormity state information of the tower.
Because the tower image collected by the collecting device carried by the flight device makes the part of the tower occupied in the tower image very small, the abnormal state of a small area can be more concerned by the second tower abnormal model through secondary detection, and the inspection accuracy is improved.
In one embodiment, the first tower anomaly model is a YOLOv4 model and/or the second tower anomaly model is a YOLOv4 model. In one embodiment, the first tower anomaly model may be a YOLOv4 model. In one embodiment, the second tower anomaly model may be a YOLOv4 model. The YOLOv4 model realizes the balance of speed and precision, and the first tower abnormity model and/or the second tower abnormity model is/are a YOLOv4 model, so that the inspection efficiency and accuracy can be improved.
Referring to fig. 4, a method for building a YOLOv4 model is shown, and as shown in fig. 4, the method for building the YOLOv4 model may include steps S402 to S412.
S402: and (4) making a tower data set, and dividing the tower data set into a tower training set and a tower testing set according to a proportion.
It can be understood that a suitable proportion can be selected according to the needs, and optionally, the proportion of the tower training set to the tower testing set is 8: 2. in one embodiment, data images along the tower are collected, position information and category information of the tower in the images are labeled, the information is stored in an xml file, pictures of the tower are screened and stored to serve as a tower data set, and the data set is divided into a tower training set and a tower testing set according to the proportion of 8: 2.
S404: and generating a corresponding index file according to the divided tower training set and tower testing set.
And constructing an index file required by model training according to the divided data set. In one embodiment, the index file may include an index file of a train set image and an index file of a test set image, and the model may be trained and tested according to the images in the index files.
S406: and clustering the pole tower training set to generate at least one tracing box with different sizes.
In one embodiment, the sizes of targets in the tower training set are clustered using k-means, obtaining 9 different sizes of anchor boxes.
S408: the YOLOv4 model was constructed.
The YOLOv4 model can be constructed in four parts: the device comprises an input end, a BackBone reference network, a Neck middle layer and a Head output layer.
And the input end of the model carries out preprocessing and data enhancement on input data. Preprocessing involves scaling the input images to meet the required size of the network, and normalizing the images to improve the speed of training and the accuracy of the network. The data enhancement comprises the steps of carrying out random erasing, translation, rotation, mirror image and random noise addition on an input image, expanding data by using Mix-up, Mosaic and SAT, and increasing the diversity of training samples and improving the training effect.
The BackBone reference network part adopts CSPDarknet53 to extract data features, the BackBone reference network part comprises 29 convolutional layers and has enough depth and receptive field, the CSPDarknet53 adds 5 CSP modules on the basis of Darknet53, the CSP divides the feature mapping of the base layer into two parts for processing, so that gradient flow is transmitted through different network paths, repeated calculation of the gradient is reduced, and then the gradient flow and the CSP are combined through a cross-layer splicing structure, and the calculated amount is reduced while the model accuracy is ensured. And by using a smooth Mish activation function, a hard zero boundary of the ReLU is avoided, and more information is allowed to enter a neural network, so that the accuracy and generalization capability of the model are improved. Dropblock is used in the model in an inserting mode, characteristics of adjacent regions are discarded, the defect that Dropout has an unobvious effect on the convolutional layer is overcome, overfitting is prevented beneficially, and generalization capability of the model is improved.
The middle layer of the Neck further extracts features from the BackBone, outputs three tensors with different scales according to the size of an object to be identified, and is used for identifying a small target, a medium target and a large target respectively. And an SPP module and an FPN + PAN structure are added to further extract the diversity of features. The SPP (spatial Pyramid Pooling networks) performs maximum Pooling by using Pooling cores with the sizes of 1, 5, 9 and 13, and then the results are spliced together to perform multi-scale fusion on the features. The FPN (feature Pyramid networks) feature Pyramid network enlarges the original feature map layer by layer from top to bottom through deconvolution to construct a feature Pyramid, which is beneficial to semantic features of objects with different sizes extracted by the network, so that the model can identify the same object with different sizes and scales. The PAN (Path Aggregation network) draws features from bottom to top by using a PANET algorithm in the field of image segmentation, and the parameters of different layers from the FPN are aggregated by a splicing method, so that the positioning capability of the network is enhanced.
The Head output layer outputs bounding boxes and corresponding classes identifying objects for three different dimensions, YOLOv4 measures the difference between the predicted and real boxes using a more effective CIOU loss function that takes into account the coverage area, center point distance, and aspect ratio of each box. The Label is smoothed when the category Label is set, using Label smooth, to prevent model overfitting.
In one embodiment, set input image size 608 x 608, batch size 8, initial learning rate 0.001, and training round 50 using exponential decay strategy. And training tower training set data by using the coco pre-training weight, and obtaining a trained network model.
S410: and testing the tower test set by using a YOLOv4 model, and acquiring a corresponding test result.
In one embodiment, the tower test set is fed into a trained YOLOv4 network for target recognition, each image will generate 22743 preselected boxes, each containing 5+ n values, which are the center point coordinates x, y and width and height w, h of the preselected box, and the confidence c and the probability of n classes. Firstly, filtering some preselected frames with low confidence degrees according to the value of the confidence degree c, wherein the confidence degree threshold value in the model is 0.5, carrying out DIOU-NMS screening on the retained preselected frames to screen out excessive repeated preselected frames, and taking the remaining preselected frames as final detection results.
S412: and comparing the detection result of the tower test set with the true value, calculating a corresponding index, and adjusting the model according to the detection result.
And comparing the detection result of the tower test set with the true value to obtain a corresponding index, analyzing the cause of the test error, and adjusting the parameters or data of the model to achieve a better effect. In one embodiment, the indicators may include a miss rate and a false rate.
Referring to fig. 5, a method for acquiring a detail image according to an embodiment of the present application is shown, and as shown in fig. 5, the method for acquiring a detail image may include steps S502 to S506.
And S502, calculating the pixel deviation between the abnormal position of the tower and the tower image.
It should be noted that the pixel deviation between the abnormal position of the tower and the tower image is calculated, that is, the distance between the abnormal position of the tower and the reference position of the tower image is calculated. In one embodiment, the pixel deviation of the abnormal position of the tower from the center of the tower image is calculated.
And S504, calculating a target angle required to be adjusted by the acquisition device according to the pixel deviation.
In one embodiment, after the angle of the acquisition device is adjusted by the target angle, the center of the image acquired by the acquisition device coincides with the abnormal position of the tower.
S506, controlling the acquisition device according to the target angle, controlling the acquisition device to focus on the abnormal position of the tower, and acquiring the detail image of the abnormal position of the tower.
In one embodiment, the angle of the camera is adjusted by controlling the three-dimensional stability augmentation platform, so that the center of a detail image acquired by the camera is the abnormal position of the tower, and the acquisition device is controlled to focus the abnormal position of the tower, so that a clear detail image of the abnormal position of the tower is acquired.
By the method for acquiring the detail image, the detail image of the abnormal position of the tower can be acquired, and the second tower abnormal model can output more accurate tower abnormal state information according to the clear detail image, so that the inspection accuracy is improved.
It can be understood that, at present, the method based on unmanned aerial vehicle on-line measuring transmits the tower image of gathering to remote server in real time, adjusts the camera angle according to data such as tower image again to aim at and wait to detect the object, but this messenger unmanned aerial vehicle hover time overlength of network time delay has reduced detection efficiency, if unmanned aerial vehicle position, angle change appear during hovering, then the adjustment data that remote server returned will lose efficacy, can make detection effect unstable.
According to the tower detection method provided by the embodiment, the pixel deviation between the abnormal position of the tower and the center of the image is determined in an edge calculation mode, the target angle required to be adjusted by the acquisition device is calculated according to the pixel deviation, the acquisition device is controlled according to the target angle, the acquisition device is controlled to focus on the abnormal position of the tower, the detail image of the abnormal position of the tower is acquired, the angle of the acquisition device can be adjusted rapidly, and the detection efficiency and the stability of the detection result are improved.
Referring to fig. 6, a tower inspection method according to a second embodiment of the present disclosure is shown, where the tower inspection method may include steps S602 to S612.
And S602, controlling the flying device to fly according to a preset inspection route.
It should be noted that the patrol route may be used to determine a flight route of the flying device, a plurality of detection points, and an order of the plurality of detection points, and the flying device flies according to the flight route, so that the collecting device collects tower images and detail images of the detection points. Optionally, a GPS positioning system is provided on the flying device. In one embodiment, the flying device is controlled to automatically patrol according to a preset patrol route and by combining with a GPS signal, and the acquisition device is controlled to acquire tower images according to a preset shooting angle when the flying device reaches a detection point. In one embodiment, if the acquired tower image does not meet the requirement, the adjusting device is adjusted according to the tower position in the tower image, so that the acquiring device acquires the tower image meeting the requirement. Optionally, the acquired tower image should include a complete tower structure, and the acquired tower image is required to include the complete tower structure.
And S604, controlling the acquisition device to acquire tower images under the condition that the flight device flies to a detection point.
And S606, determining the abnormal position of the tower according to the tower image and the first tower abnormal model.
In one embodiment, if the abnormal position of the tower is determined according to the tower image and the first tower abnormal model, the flying device is controlled to fly to the next detection point according to a preset routing inspection route.
And S608, controlling the acquisition device to acquire the detail image of the abnormal position of the tower according to the abnormal position of the tower.
The acquisition device is arranged on the flight device and used for acquiring tower images.
And S610, determining abnormal state information of the tower according to the detail image and the second tower abnormal model.
And S612, sending the abnormal state information to the terminal equipment.
Referring to fig. 7, a tower inspection method according to a third embodiment of the present disclosure is shown, where the tower inspection method may include steps S702 to S710.
And S702, controlling the flight device to automatically patrol according to a preset patrol route by combining a GPS signal, controlling the acquisition device to acquire a tower image according to a preset shooting angle when reaching a detection point, and adjusting the angle of the holder according to the position of the tower in the tower image so as to finally align the acquisition device to the tower.
And S704, storing the acquired tower image on a development plate, detecting whether an abnormal position exists in the tower image by using a first tower abnormal model deployed on the development plate, if not, executing the step S702, and if so, executing the step S706.
The first tower abnormity model is a YOLOv4 model.
S706, calculating the pixel deviation of the abnormal position between the position of the tower image and the center of the tower image, calculating the angle of the camera to be adjusted according to the pixel deviation, then controlling the three-dimensional stability augmentation platform to adjust the angle of the camera, and controlling the camera to focus on the abnormal position of the tower to obtain a clear detail image of the abnormal position of the tower.
And S708, detecting abnormal state information of the abnormal position in the detail image by using the second tower abnormal model on the development board.
And the second tower abnormity model is a YOLOv4 model.
And S710, recording the detail image and the current abnormal state information of the abnormal position on the development board, and transmitting the detail image and the current abnormal state information to the flat board.
Wherein, the flat board is used by the staff, makes things convenient for the staff to monitor patrolling and examining.
According to the pole tower inspection system provided by the embodiment, the information transmission does not influence the inspection flow of the unmanned aerial vehicle, and if the detection effect cannot be influenced due to the fact that the information cannot be transmitted due to network reasons, the staff can wait for the unmanned aerial vehicle to finish copying the detection result after inspection, so that the applicability of the pole tower inspection method is improved.
It should be understood that although the various steps in the flowcharts of fig. 2-7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-7 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 8, a pole tower inspection device 800 is provided, comprising: an obtaining module 802, a determining module 804, and a sending module 806. The obtaining module 802 is configured to obtain a tower image. The determining module 804 is configured to determine abnormal state information of the tower according to the tower image and the tower abnormal model. The sending module 806 is configured to send the abnormal state information to the terminal device.
In one embodiment, the determining module may be further configured to determine an abnormal position of the tower according to the tower image and the first tower abnormal model; the determining module can also be used for controlling the collecting device to obtain a detailed image of the abnormal position of the tower according to the abnormal position of the tower. The acquisition device is used for acquiring images of the tower; the determining module can also be used for determining the abnormal state information of the tower according to the detail image and the second tower abnormal model.
In one embodiment, the determination module may be further configured to calculate a pixel deviation between the abnormal position of the tower and the tower image. The determining module can also be used for calculating a target angle required to be adjusted by the acquisition device according to the pixel deviation. The determining module can also be used for controlling the collecting device according to the target angle, controlling the collecting device to focus on the abnormal position of the tower and collecting the detail image of the abnormal position of the tower.
In one embodiment, the tower inspection device may further include a first control module and a second control module. Wherein, first control module can be used for controlling flying device to patrol and examine the route flight according to predetermineeing. Wherein, the flying device is provided with a collection device. The second control module can be used for controlling the acquisition device to acquire tower images under the condition that the flying device flies to the detection point.
In one embodiment, the tower inspection device may further include a third control module. The third control module can be used for controlling the flying device to fly to the next detection point according to the preset routing inspection route.
For specific limitations of the tower inspection device, reference may be made to the above limitations on the tower inspection method, and details are not repeated here. All modules in the tower inspection device can be completely or partially realized through software, hardware and a combination of the software and the hardware. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.
In an embodiment, the tower inspection system described in the above embodiment may further include a storage device, where a computer program is stored in the storage device, and the control device implements the steps in each of the tower inspection method embodiments when executing the computer program.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the tower patrol method embodiments described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
In the description herein, references to the description of "some embodiments," "other embodiments," "desired embodiments," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, a schematic description of the above terminology may not necessarily refer to the same embodiment or example.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. The utility model provides a shaft tower system of patrolling and examining which characterized in that includes:
a flying device;
the acquisition device is arranged on the flying device and used for acquiring the tower image;
the control device is arranged on the flying device, is connected with the acquisition device, is used for acquiring tower images, is also used for determining abnormal state information of the tower according to the tower images and the tower abnormal model, and is also used for sending the abnormal state information;
and the terminal equipment is connected with the control device and used for receiving the abnormal state information sent by the control device.
2. The pole tower inspection system according to claim 1, further comprising an adjustment device for adjusting a collection angle of the collection device.
3. A pole tower inspection method is characterized by comprising the following steps:
acquiring a tower image;
determining abnormal state information of the tower according to the tower image and the tower abnormal model;
and sending the abnormal state information to terminal equipment.
4. The pole tower inspection method according to claim 3, wherein the step of determining abnormal state information of the pole tower according to the pole tower image and the pole tower abnormal model comprises the steps of:
determining the abnormal position of the tower according to the tower image and the first tower abnormal model;
controlling an acquisition device to acquire a detail image of the abnormal position of the tower according to the abnormal position of the tower; the acquisition device is used for acquiring images of the tower;
and determining the abnormal state information of the tower according to the detail image and the second tower abnormal model.
5. The pole tower inspection method according to claim 4, wherein the step of controlling the acquisition device to acquire the detailed image of the abnormal position of the pole tower according to the abnormal position of the pole tower comprises the following steps:
calculating the pixel deviation between the abnormal position of the tower and the tower image;
calculating a target angle required to be adjusted by the acquisition device according to the pixel deviation;
and controlling the acquisition device according to the target angle, controlling the acquisition device to focus on the abnormal position of the tower, and acquiring a detail image of the abnormal position of the tower.
6. The pole and tower inspection method according to claim 4, further comprising:
controlling a flying device to fly according to a preset inspection route; wherein the flying device is provided with the collecting device;
and controlling the acquisition device to acquire the tower image under the condition that the flying device flies to a detection point.
7. The pole and tower inspection method according to claim 6, further comprising:
and if the abnormal position of the tower is determined according to the tower image and the first tower abnormal model, controlling the flying device to fly to the next detection point according to the preset routing inspection route.
8. The pole tower inspection method according to any one of claims 4 to 7, wherein the first pole tower abnormality model is a YOLOv4 model, and/or the second pole tower abnormality model is a YOLOv4 model.
9. The utility model provides a shaft tower inspection device which characterized in that includes:
the acquisition module is used for acquiring a tower image;
the determining module is used for determining the abnormal state information of the tower according to the tower image and the tower abnormal model;
and the sending module is used for sending the abnormal state information to the terminal equipment.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the pole tower inspection method according to any one of claims 3 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111308904.7A CN114035606A (en) | 2021-11-05 | 2021-11-05 | Pole tower inspection system, pole tower inspection method, control device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111308904.7A CN114035606A (en) | 2021-11-05 | 2021-11-05 | Pole tower inspection system, pole tower inspection method, control device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114035606A true CN114035606A (en) | 2022-02-11 |
Family
ID=80143123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111308904.7A Pending CN114035606A (en) | 2021-11-05 | 2021-11-05 | Pole tower inspection system, pole tower inspection method, control device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114035606A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN117472069A (en) * | 2023-12-28 | 2024-01-30 | 烟台宇控软件有限公司 | Robot control method and system for power transmission line detection |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105892485A (en) * | 2016-04-13 | 2016-08-24 | 重庆雅智兰能源管理有限公司 | Overhead line inspection system |
| WO2017024975A1 (en) * | 2015-08-07 | 2017-02-16 | 清华大学深圳研究生院 | Unmanned aerial vehicle portable ground station processing method and system |
| CN108039680A (en) * | 2017-11-13 | 2018-05-15 | 国网山东省电力公司荣成市供电公司 | A kind of automatic cruising power circuit detection unmanned plane and its detection method |
| CN108377328A (en) * | 2018-01-03 | 2018-08-07 | 广东电网有限责任公司机巡作业中心 | A kind of helicopter makes an inspection tour the target image pickup method and device of operation |
| CN110633629A (en) * | 2019-08-02 | 2019-12-31 | 广东电网有限责任公司清远供电局 | Power grid inspection method, device, equipment and storage medium based on artificial intelligence |
| CN111080832A (en) * | 2019-12-27 | 2020-04-28 | 上海复亚智能科技有限公司 | Inspection method and system for power transmission line tower |
| CN111966133A (en) * | 2020-08-29 | 2020-11-20 | 山东翔迈智能科技有限公司 | Visual servo control system of holder |
| CN112560586A (en) * | 2020-11-27 | 2021-03-26 | 国家电网有限公司大数据中心 | Method and device for obtaining structured data of pole and tower signboard and electronic equipment |
| CN112799422A (en) * | 2021-04-06 | 2021-05-14 | 众芯汉创（北京）科技有限公司 | Unmanned aerial vehicle flight control method and device for power inspection |
| CN112990335A (en) * | 2021-03-31 | 2021-06-18 | 江苏方天电力技术有限公司 | Intelligent recognition self-learning training method and system for power grid unmanned aerial vehicle inspection image defects |
| CN113297915A (en) * | 2021-04-28 | 2021-08-24 | 江苏师范大学 | Insulator recognition target detection method based on unmanned aerial vehicle inspection |
| CN113419564A (en) * | 2021-08-24 | 2021-09-21 | 天津市普迅电力信息技术有限公司 | Power channel inspection method based on fuzzy path |
| CN113485433A (en) * | 2021-07-27 | 2021-10-08 | 国网青海省电力公司海西供电公司 | Inspection method and device for power transmission line |
| CN113486865A (en) * | 2021-09-03 | 2021-10-08 | 国网江西省电力有限公司电力科学研究院 | Power transmission line suspended foreign object target detection method based on deep learning |
-
2021
- 2021-11-05 CN CN202111308904.7A patent/CN114035606A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017024975A1 (en) * | 2015-08-07 | 2017-02-16 | 清华大学深圳研究生院 | Unmanned aerial vehicle portable ground station processing method and system |
| CN105892485A (en) * | 2016-04-13 | 2016-08-24 | 重庆雅智兰能源管理有限公司 | Overhead line inspection system |
| CN108039680A (en) * | 2017-11-13 | 2018-05-15 | 国网山东省电力公司荣成市供电公司 | A kind of automatic cruising power circuit detection unmanned plane and its detection method |
| CN108377328A (en) * | 2018-01-03 | 2018-08-07 | 广东电网有限责任公司机巡作业中心 | A kind of helicopter makes an inspection tour the target image pickup method and device of operation |
| CN110633629A (en) * | 2019-08-02 | 2019-12-31 | 广东电网有限责任公司清远供电局 | Power grid inspection method, device, equipment and storage medium based on artificial intelligence |
| CN111080832A (en) * | 2019-12-27 | 2020-04-28 | 上海复亚智能科技有限公司 | Inspection method and system for power transmission line tower |
| CN111966133A (en) * | 2020-08-29 | 2020-11-20 | 山东翔迈智能科技有限公司 | Visual servo control system of holder |
| CN112560586A (en) * | 2020-11-27 | 2021-03-26 | 国家电网有限公司大数据中心 | Method and device for obtaining structured data of pole and tower signboard and electronic equipment |
| CN112990335A (en) * | 2021-03-31 | 2021-06-18 | 江苏方天电力技术有限公司 | Intelligent recognition self-learning training method and system for power grid unmanned aerial vehicle inspection image defects |
| CN112799422A (en) * | 2021-04-06 | 2021-05-14 | 众芯汉创（北京）科技有限公司 | Unmanned aerial vehicle flight control method and device for power inspection |
| CN113297915A (en) * | 2021-04-28 | 2021-08-24 | 江苏师范大学 | Insulator recognition target detection method based on unmanned aerial vehicle inspection |
| CN113485433A (en) * | 2021-07-27 | 2021-10-08 | 国网青海省电力公司海西供电公司 | Inspection method and device for power transmission line |
| CN113419564A (en) * | 2021-08-24 | 2021-09-21 | 天津市普迅电力信息技术有限公司 | Power channel inspection method based on fuzzy path |
| CN113486865A (en) * | 2021-09-03 | 2021-10-08 | 国网江西省电力有限公司电力科学研究院 | Power transmission line suspended foreign object target detection method based on deep learning |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN117472069A (en) * | 2023-12-28 | 2024-01-30 | 烟台宇控软件有限公司 | Robot control method and system for power transmission line detection |
| CN117472069B (en) * | 2023-12-28 | 2024-03-26 | 烟台宇控软件有限公司 | Robot control method and system for power transmission line detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| CN112380952B (en) | Power equipment infrared image real-time detection and identification method based on artificial intelligence | |
| JP2022514974A (en) | Object detection methods, devices, electronic devices, and computer programs | |
| CN110348304A (en) | A kind of maritime affairs distress personnel search system being equipped on unmanned plane and target identification method | |
| CN114035606A (en) | Pole tower inspection system, pole tower inspection method, control device and storage medium | |
| CN111524132B (en) | Method, device and storage medium for identifying abnormal cells in sample to be detected | |
| CN113298035A (en) | Unmanned aerial vehicle electric power tower detection and autonomous cruise method based on image recognition | |
| CN108932273A (en) | Picture screening technique and device | |
| CN113205116A (en) | Automatic extraction and flight path planning method for unmanned aerial vehicle inspection shooting target point of power transmission line | |
| CN113486779A (en) | Panoramic intelligent inspection system for power transmission line | |
| CN115793689A (en) | Unmanned aerial vehicle automatic overhead transmission line inspection method and system based on front-end target identification | |
| CN116719339A (en) | Unmanned aerial vehicle-based power line inspection control method and system | |
| CN114020043A (en) | Unmanned aerial vehicle building project supervision system and method, electronic equipment and storage medium | |
| CN113313006A (en) | Urban illegal construction supervision method and system based on unmanned aerial vehicle and storage medium | |
| CN116310891A (en) | Cloud-edge cooperative transmission line defect intelligent detection system and method | |
| CN113706496A (en) | Aircraft structure crack detection method based on deep learning model | |
| CN113313110A (en) | License plate type recognition model construction and license plate type recognition method | |
| CN111738312B (en) | Power transmission line state monitoring method and device based on GIS and virtual reality fusion and computer readable storage medium | |
| CN112631333A (en) | Target tracking method and device of unmanned aerial vehicle and image processing chip | |
| CN112529836A (en) | High-voltage line defect detection method and device, storage medium and electronic equipment | |
| CN114659499B (en) | Smart city 3D map model photography establishment method based on unmanned aerial vehicle technology | |
| JP2014099055A (en) | Detector, detection method, and program | |
| CN114037895A (en) | Unmanned aerial vehicle pole tower inspection image identification method | |
| CN112734794B (en) | Moving target tracking and positioning method based on deep learning | |
| CN115482277A (en) | Social distance risk early warning method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |