CN115909096A

CN115909096A - Unmanned aerial vehicle cruise pipeline hidden danger analysis method, device and system

Info

Publication number: CN115909096A
Application number: CN202211351100.XA
Authority: CN
Inventors: 龙莉; 黎玉清; 林亮亮; 刘健; 汤坚; 范亮; 劳健华; 李志强
Original assignee: Guangzhou Zhongke Zhi Tour Technology Co ltd; South China Blue Sky Aviation Oil Co ltd Hunan Branch
Current assignee: Guangzhou Zhongke Zhi Tour Technology Co ltd; South China Blue Sky Aviation Oil Co ltd Hunan Branch
Priority date: 2022-10-31
Filing date: 2022-10-31
Publication date: 2023-04-04

Abstract

The application provides an unmanned aerial vehicle cruise pipeline hidden danger analysis method, device and system. Wherein the method comprises the following steps: acquiring pre-generated route information, wherein the route information comprises a route of the unmanned aerial vehicle and flight parameter information; the route comprises a plurality of waypoints, and the waypoints are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area; automatically patrolling by using an unmanned aerial vehicle according to the route information to obtain a patrol video of the target oil pipeline; inputting each image in the inspection video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image; the hidden danger analysis result comprises a hidden danger category and a hidden danger position, and the hidden danger category comprises: a building category, a construction site category, and a construction machine category; and determining whether the potential safety hazard exists in the target aviation fuel pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and the preset region of interest.

Description

Unmanned aerial vehicle cruise pipeline hidden danger analysis method, device and system

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a method, a device and a system for analyzing hidden dangers of cruise pipelines of an unmanned aerial vehicle.

Background

Along with the development of unmanned aerial vehicle inspection technology, unmanned aerial vehicle more and more is applied to the daily work of patrolling and examining of boat oil pipeline.

At present, an unmanned aerial vehicle is utilized to patrol and examine an aviation oil pipeline, the flying patrol task of a set route is mostly completed through a mode of manually controlling the unmanned aerial vehicle, and then, the unmanned aerial vehicle patrol and examine videos are spliced and processed manually so as to identify the potential safety hazard of the aviation oil pipeline. However, in the process of identifying the potential safety hazard, a large amount of manpower and time cost are required to complete flight control and data processing, the task amount of the manual flight inspection task is large, and the potential safety hazard existing in the aviation oil pipeline cannot be identified in time.

Disclosure of Invention

The application provides an unmanned aerial vehicle cruise pipeline potential safety hazard analysis method, device and system, and identifies the potential safety hazard of an aviation oil pipeline in time.

The application provides an unmanned aerial vehicle cruise pipeline hidden danger analysis method, the method includes:

acquiring pre-generated route information, wherein the route information comprises a route of the unmanned aerial vehicle and flight parameter information; the flight line comprises a plurality of flight points, and the flight points are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area;

automatically patrolling by using an unmanned aerial vehicle according to the route information to obtain a patrol video of the target oil pipeline;

inputting each image in the inspection video into a hidden danger analysis model one by one so as to output a hidden danger analysis result of each image; the hidden danger analysis model is obtained by training hidden danger results of sample images and the sample images, the hidden danger analysis results comprise hidden danger categories and hidden danger positions, and the hidden danger categories comprise: a building category, a construction site category, and a construction machine category;

and determining whether the potential safety hazard exists in the target aviation oil pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and the preset region of interest.

Further, the method further comprises:

determining an area within a preset range around a target aviation oil pipeline as a patrol area;

acquiring point cloud data of the inspection area to obtain target point cloud data of the target oil pipeline;

generating a three-dimensional point cloud model of the target oil and gas pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of a target ground object in the inspection area, which are acquired in advance;

determining a plurality of navigation points based on points with the height greater than a preset threshold value of the ground object coordinate in the three-dimensional point cloud model;

connecting the multiple navigation points in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline;

setting flight parameter information for the route to obtain route information; the flight parameter information includes at least one of a flight speed, a shooting distance, and a shooting attitude.

Further, the hidden danger analysis model is a target detection model based on YOLOv4, the hidden danger analysis model comprises an input module, a backbone network, a neck network and an output module, the backbone network adopts a CSPDarknet53 structure, and the neck network comprises a spatial pyramid pooling SPP network module, a feature pyramid network FPN structure and a path aggregation network PAN structure; training the hidden danger analysis model in the following way:

acquiring a plurality of sample images;

marking the hidden danger category and the hidden danger position of the target aviation oil pipeline in the sample image to obtain a sample data set;

determining training sample data from the sample data set, wherein the training sample data comprises a plurality of first sample images and hidden danger categories and hidden danger positions of the first sample images;

inputting each first sample image into the input module for image preprocessing to obtain a preprocessed image;

inputting each preprocessed image into the backbone network for feature extraction so as to output a first feature map of a first sample image corresponding to the preprocessed image;

inputting each first feature map into the neck network for feature extraction so as to output a second feature map of the first sample image;

inputting each second feature map into the output module, and calculating the category and the position of each second feature map to obtain the predicted hidden danger category and the predicted hidden danger position of the first sample image;

calculating the current loss according to the predicted hidden danger category and the predicted hidden danger position of the first sample image and the hidden danger category and the hidden danger position of the first sample image;

and adjusting the training parameters of the hidden danger analysis model according to the current loss until a preset ending condition is met, and obtaining the trained hidden danger analysis model.

Further, the inputting each first sample image into the input module for image preprocessing to obtain a preprocessed image includes:

inputting each first sample image into the input module to perform size normalization processing on each first sample image to obtain a normalized image;

and performing data enhancement processing on each normalized image to obtain a preprocessed image.

Further, after the hidden danger category and the hidden danger position of the target aviation oil pipeline in the sample image are labeled to obtain a sample data set, the method further comprises the following steps:

determining test sample data from the sample data set, wherein the test sample data comprises a plurality of second sample images and the hidden danger categories and hidden danger positions of the second sample images;

inputting each second sample image into a trained hidden danger analysis model to output the predicted hidden danger category and the predicted hidden danger position of the second sample image;

calculating the omission factor and the false alarm factor of the hidden danger analysis model according to the predicted hidden danger category and the predicted hidden danger position of the second sample image and the hidden danger category and the hidden danger position of the second sample image;

adjusting training parameters of the hidden danger analysis model according to the current loss until a preset ending condition is met to obtain a trained hidden danger analysis model, wherein the method comprises the following steps of:

and adjusting the training parameters of the hidden danger analysis model according to the current loss and the missing detection rate and the false alarm rate of the hidden danger analysis model until the preset ending condition is met, and obtaining the trained hidden danger analysis model when the missing detection rate and the false alarm rate of the hidden danger analysis model meet the preset conditions.

for each marked sample image in the sample data set, clustering a marking frame corresponding to the hidden danger position based on the hidden danger position of the marked target aviation oil pipeline in the marked sample image to obtain a clustering result corresponding to the hidden danger position;

and updating the position of the hidden danger according to the clustering result.

Further, the current loss of the hidden danger analysis model is determined according to the intersection ratio of the hidden danger prediction frame and the hidden danger real frame of the first sample image, the distance of the central point and the length-width ratio. The application provides an unmanned aerial vehicle pipeline hidden danger analytical equipment that cruises, the device includes:

the information acquisition unit is used for acquiring pre-generated route information, wherein the route information comprises a route of the unmanned aerial vehicle and flight parameter information; the flight path comprises a plurality of flight points, and the flight points are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area;

the route patrol unit is used for carrying out automatic patrol by utilizing an unmanned aerial vehicle according to the route information to obtain a patrol video of the target aviation oil pipeline;

the hidden danger analysis unit is used for inputting each image in the inspection video into a hidden danger analysis model one by one so as to output a hidden danger analysis result of each image; the hidden danger analysis model is obtained by training hidden danger results of sample images, the hidden danger analysis results comprise hidden danger categories and hidden danger positions, and the hidden danger categories comprise: a building category, a construction site category, and a construction machine category;

and the hidden danger determining unit is used for determining whether the potential safety hazard exists in the target aviation fuel pipeline at the corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and the preset interested region.

The application provides an unmanned aerial vehicle pipeline hidden danger analytic system that cruises, the system includes: the system comprises an air route planning module, an air route inspection module and a hidden danger analysis platform;

the route planning module is used for determining an area in a preset range around the target aviation oil pipeline as a patrol area; acquiring point cloud data of the inspection area to obtain target point cloud data of the target oil pipeline; generating a three-dimensional point cloud model of the target oil and gas pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of a target ground object in the inspection area, which are acquired in advance; determining a plurality of navigation points based on points in the three-dimensional point cloud model, the height of which is greater than a preset threshold value of ground object coordinates; connecting the multiple navigation points in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline; setting flight parameter information for the route to obtain route information; the flight parameter information comprises at least one of flight speed, shooting distance and shooting attitude;

the route patrol module is used for automatically patrolling by using an unmanned aerial vehicle according to the route information to obtain a patrol video of the target oil and gas pipeline and sending the patrol video to the hidden danger analysis platform;

the hidden danger analysis platform is used for inputting each image in the inspection video into a hidden danger analysis model one by one so as to output a hidden danger analysis result of each image; determining whether potential safety hazards exist in a target aviation oil pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and a preset region of interest, and generating a potential safety hazard analysis report under the condition that the potential safety hazards exist in the target aviation oil pipeline; the hidden danger analysis model is obtained by training hidden danger results of sample images, the hidden danger analysis results comprise hidden danger categories and hidden danger positions, and the hidden danger categories comprise: a building category, a construction site category, and a construction machine category.

An electronic device includes a processor and a memory;

a memory for storing a computer program;

and the processor is used for realizing the unmanned aerial vehicle cruising pipeline hidden danger analysis method when the program stored in the memory is executed.

The application provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the method for analyzing the hidden danger of the cruise pipeline of the unmanned aerial vehicle is implemented.

According to the method, the device and the system for analyzing the hidden danger of the cruising pipeline of the unmanned aerial vehicle, pre-generated route information including the route of the unmanned aerial vehicle and flight parameter information can be obtained, a plurality of waypoints in the route are determined based on the three-dimensional point cloud model of the cruising area of the target cruising oil pipeline, and the unmanned aerial vehicle is used for automatically cruising according to the route information to obtain the cruising video of the target cruising oil pipeline. Therefore, manual flight inspection is not needed, automatic inspection of the aviation oil pipeline is achieved, and labor and time costs are reduced. And inputting each image in the inspection video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image, and determining whether the potential safety hazard exists in the target aviation oil pipeline at the corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and the preset region of interest. Therefore, the potential safety hazard of the aviation oil pipeline can be identified in time based on automatic inspection and automatic analysis of the potential safety hazard of the aviation oil pipeline.

Drawings

Fig. 1 is a schematic diagram illustrating an embodiment of an unmanned aerial vehicle cruise pipeline route planning according to an embodiment of the present application;

FIG. 2 is a schematic view of a three-dimensional point cloud model of an oil pipeline according to an embodiment of the present disclosure;

FIG. 3 is a schematic illustration of a route display corresponding to the oil pipeline of the present application;

FIG. 4 is a schematic view illustrating route information corresponding to a navigation oil pipeline according to an embodiment of the present disclosure;

fig. 5 is a flowchart illustrating an unmanned aerial vehicle cruise pipeline hidden danger analysis method according to an embodiment of the present application;

FIG. 6a is a schematic view illustrating a predetermined region of interest according to an embodiment of the present application;

fig. 6b is a schematic view illustrating a hidden danger position according to an embodiment of the present application;

fig. 7 is a schematic diagram illustrating a hidden danger analysis model training method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an unmanned aerial vehicle cruise pipeline hidden danger analysis device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an unmanned aerial vehicle cruise pipeline hidden danger analysis system according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the exemplary embodiments below do not represent all embodiments consistent with one or more embodiments of the specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of one or more embodiments of the specification, as detailed in the claims which follow.

It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described herein. In some other embodiments, the method may include more or fewer steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.

In order to timely identify potential safety hazards of an oil pipeline, the embodiment of the application provides a method, a device and a system for analyzing potential safety hazards of a cruise pipeline of an unmanned aerial vehicle.

In one embodiment of the application, an unmanned aerial vehicle cruise pipeline hidden danger analysis method is provided, which comprises the following steps:

acquiring pre-generated route information, wherein the route information comprises routes of the unmanned aerial vehicle and flight parameter information; the flight path comprises a plurality of flight points, and the flight points are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area;

inputting each image in the inspection video into a hidden danger analysis model one by one so as to output a hidden danger analysis result of each image; the hidden danger analysis model is obtained by training hidden danger results of sample images, the hidden danger analysis results comprise hidden danger categories and hidden danger positions, and the hidden danger categories comprise: a building category, a construction site category, and a construction machine category;

The method for analyzing the hidden danger of the cruising pipeline of the unmanned aerial vehicle can acquire pre-generated route information comprising the route of the unmanned aerial vehicle and flight parameter information, a plurality of route points in the route are determined based on a three-dimensional point cloud model of the cruising area of the target aviation oil pipeline, and the unmanned aerial vehicle is used for automatically cruising according to the route information to obtain the cruising video of the target aviation oil pipeline. So, do not need artifical flight to patrol, realize the automation of aviation oil pipeline and patrol, reduce manpower and time cost. And inputting each image in the inspection video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image, and determining whether the potential safety hazard exists in the target aviation oil pipeline at the corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and the preset region of interest. Therefore, the potential safety hazard of the aviation oil pipeline can be identified in time based on automatic inspection and automatic analysis of the potential safety hazard of the aviation oil pipeline, the identification efficiency of the potential safety hazard of the aviation oil pipeline is improved, and the labor and time cost is reduced.

In the correlation technique, the aviation oil pipeline can carry out unmanned aerial vehicle patrol once every month in the daily patrol maintenance process, generate a corresponding air route before patrol, set a fixed flight height for the air route, and then complete the flight patrol task of the generated air route by manually controlling the unmanned aerial vehicle. In the process, in order to avoid high points such as high mountains and high-voltage lines, the set flying height is high, and therefore a lower place cannot be clearly patrolled.

Based on this, referring to fig. 1, an unmanned aerial vehicle cruise pipeline automatic route planning method provided in an embodiment of the present application may include the following steps S101 to S106:

and S101, determining an area in a preset range around the target aviation oil pipeline as a patrol area.

In one example, the preset range around the target oil pipeline may be a preset width area on each side of the target oil pipeline, and the preset width may be set according to actual requirements, such as 50 meters, 100 meters, 150 meters, or the like. For example, the area within 100 meters of each side of the target marine oil pipeline is determined as the patrol area.

S102, collecting point cloud data of the inspection area to obtain target point cloud data of the target aviation oil pipeline.

In one example, laser three-dimensional scanning is performed on three-dimensional terrain information such as terrain, ground objects and the like in an inspection area of a target aviation fuel pipeline in a mode that an unmanned aerial vehicle carries a laser radar, and target point cloud data of the target aviation fuel pipeline is obtained.

Preferably, after the target point cloud data of the target oil and gas pipeline is obtained, noise in the target point cloud data can be further optimized, so that the influence of the noise in the target point cloud data on the accuracy of air route planning is avoided.

And S103, generating a three-dimensional point cloud model of the target oil and gas pipeline based on the target point cloud data and preset control point coordinates.

And the control point coordinates are real coordinates of target ground objects in the inspection area collected in advance.

In one example, a high-precision measuring instrument such as an RTK (Real-time kinematic) instrument may be used to acquire the spatial coordinates of the target ground object in the inspection area of the target oil and gas pipeline, and the acquired spatial coordinates may be used as the coordinates of the control point. Wherein, the target surface feature can be confirmed according to actual demand, for example, the surface feature such as high tower, landmark in the preset range around the target aviation fuel pipeline, and this preset range can be the same with target aviation fuel pipeline tour regional.

The control points are used for converting corresponding point cloud coordinates in the target point cloud data into points in real space three-dimensional coordinates. After the control point coordinates are obtained, the control point coordinates are imported into target point cloud data of the target oil-filling pipeline, so that the space coordinates of each point cloud data in the target point cloud data are corrected, and a three-dimensional point cloud model of the target oil-filling pipeline with space coordinate information is generated.

Illustratively, the generated three-dimensional point cloud model of the target oil pipeline is shown in fig. 2, wherein different color depths in fig. 2 represent different heights.

And S104, determining a plurality of navigation points based on the points with the height higher than the preset threshold value of the ground object coordinates in the three-dimensional point cloud model.

In order to avoid the problem that a lower place cannot be clearly patrolled due to the fact that a high point is avoided to set a fixed flying height, in the embodiment of the application, coordinates such as a mountain and a high-voltage line in a three-dimensional topographic map can be accurately obtained due to laser three-dimensional scanning, and then, waypoints with different heights can be set based on the point, in the three-dimensional point cloud model, of which the height is larger than a preset threshold value of a ground object coordinate. The preset threshold value can be set according to the actual terrain of the area where the target aviation fuel pipeline is located.

For example, the preset threshold is set to 50 meters, and then, all points with a height greater than 50 meters of the corresponding ground object coordinate on the ground may be determined in the three-dimensional point cloud model, and a plurality of points with the same or different horizontal distances may be selected from the points and determined as waypoints.

And S105, connecting a plurality of navigation points in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline.

For example, a plurality of waypoints in the three-dimensional point cloud model are connected, and the generated route corresponding to the target oil pipeline is shown in fig. 3. The waypoint 1 shown in the map of the three-dimensional point cloud model in fig. 3 corresponds to the feature 1, the waypoint 2 corresponds to the feature 2, the waypoint 3 corresponds to the feature 3, the waypoint 4 corresponds to the feature 4, and the waypoint 5 corresponds to the feature 5.

And S106, setting flight parameter information for the air route to obtain air route information.

After a route corresponding to the target aviation oil pipeline is generated, unmanned aerial vehicle flight parameter information including at least one of flight speed, shooting distance and shooting attitude is set for the route, and route information is obtained. The shooting attitude may be, for example, a shooting angle, a shooting motion, or the like.

For example, the obtained route information is shown in fig. 4, S in fig. 4 represents a starting waypoint, data at each waypoint represents the flight height of the drone, and the predicted time for the drone to fly can be calculated according to the flight speed of the drone and the length of the route.

In the embodiment of the application, point cloud data of a target oil and gas pipeline inspection area are collected to obtain target point cloud data of the target oil and gas pipeline, then a three-dimensional point cloud model of the target oil and gas pipeline is generated based on the target point cloud data and preset control point coordinates, furthermore, a plurality of waypoints with different heights are determined based on points with the heights larger than preset threshold values of ground object coordinates in the three-dimensional point cloud model, a route corresponding to the target oil and gas pipeline is generated, flight parameter information is set for the route, route information is obtained, automatic route planning of the unmanned aerial vehicle inspection pipeline is achieved, the determined route can change along with the terrain because the waypoints are determined based on the points with the heights larger than the preset threshold values of the ground object coordinates in the three-dimensional point cloud model, and the problem that inspection cannot be clearly conducted at lower places caused by the fact that fixed flight heights are set for avoiding the high points is solved.

After the route information is determined, the unmanned aerial vehicle can be used for automatically patrolling according to the determined route information to obtain a patrol video of the target aviation oil pipeline, and hidden danger analysis is further carried out on the target aviation oil pipeline based on the patrol video. For example, after determining the route information, the route information may be imported into a control system of the drone, which may be, for example, a Pilot APP (flight Application), and the drone may be used to perform automatic patrol according to the route information.

Referring to fig. 5, the embodiment of the application provides an unmanned aerial vehicle cruise pipeline hidden danger analysis method, which can be applied to scenes such as oil and gas pipeline potential safety hazard analysis, and the method can include the following steps S501 to S504:

s501, pre-generated route information is obtained. The process of generating the pre-generated route information may refer to the implementation process of the embodiment shown in fig. 1, the acquired route information may include a route of the unmanned aerial vehicle and flight parameter information, the route includes a plurality of waypoints, the waypoints are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area, and the flight parameter information includes at least one of a flight speed, a shooting distance, and a shooting attitude.

And S502, automatically patrolling by using the unmanned aerial vehicle according to the route information to obtain a patrol video of the target aviation oil pipeline.

And S503, inputting each image in the patrol video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image.

After the patrol video of the target oil and gas pipeline is obtained, inputting each frame of image in the patrol video into the hidden danger analysis model one by one for hidden danger analysis, so as to obtain a hidden danger analysis result of each image output by the hidden danger analysis model. The hidden danger analysis model is obtained by training hidden danger results of sample images, the hidden danger analysis results can comprise hidden danger categories, hidden danger positions and the like, and the hidden danger categories can comprise: building type, construction site type, construction machine type, and the like.

In an example, the position of the hidden danger in the obtained hidden danger analysis result of each image may be coordinate information, size information and the like of the hidden danger in the image, and the coordinate information can clearly know the position of the hidden danger in the image. For example, the position of the hidden danger in the image may be represented in the image by using a hidden danger detection box.

S504, determining whether potential safety hazards exist in the target aviation oil pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and the preset region of interest.

Utilize unmanned aerial vehicle to carry out automatic tour according to the airline information, the tour scope that the tour video of target aviation oil pipeline that obtains corresponds is wider, the hidden danger of discerning in this tour scope, some can cause the real hidden danger of threat to target aviation oil pipeline, and can not cause real threat to target aviation oil pipeline some, and then not real hidden danger, this moment, can set for the hidden danger region that can cause real threat to target aviation oil pipeline, and then accurate judgement which is real hidden danger that causes the threat to target aviation oil pipeline.

After a hidden danger analysis result of each image in the inspection video is obtained by using the hidden danger analysis model, whether the hidden danger in the identified image is the real hidden danger of the target aviation oil pipeline is further judged.

The preset region of interest may be a hidden danger region that can pose a real threat to the target aviation fuel pipeline, in one example, the preset region of interest is preset regions on the left and right sides of the target aviation fuel pipeline, and the specific region range may be determined according to a region, a terrain, and the like where the target aviation fuel pipeline is located. For example, the area within a range of 3 meters, 5 meters or 8 meters on the left and right sides of the target oil pipeline is determined as the preset region of interest. For example, as shown in fig. 6a, a middle dotted line in fig. 6a represents a target oil pipeline, solid lines on both sides of the dotted line represent boundary lines of a preset region of interest, and a region within the solid line is the preset region of interest.

After the hidden danger analysis result of each image in the patrol video is obtained by using the hidden danger analysis model, whether intersection exists between the hidden danger position in the hidden danger analysis result of each image and a preset interested region is further judged, if the intersection exists, the potential safety hazard exists in the target aviation oil pipeline at the corresponding position of the image is judged, and if the intersection does not exist, the potential safety hazard does not exist in the target aviation oil pipeline at the corresponding position of the image.

Illustratively, as shown in fig. 6b, a hidden danger position (indicated by a block in fig. 6 b) in one of the images is shown in fig. 6b, if an intersection exists between the block in fig. 6b and the preset region of interest in fig. 6a, it is determined that a potential safety hazard exists in the target aviation fuel pipeline at the corresponding position of the image, otherwise, it is determined that the potential safety hazard does not exist in the target aviation fuel pipeline at the corresponding position of the image.

The method for analyzing the hidden danger of the cruising pipeline of the unmanned aerial vehicle can acquire pre-generated route information comprising the route of the unmanned aerial vehicle and flight parameter information, a plurality of route points in the route are determined based on a three-dimensional point cloud model of the cruising area of the target cruising pipeline, the unmanned aerial vehicle is used for automatically cruising according to the route information to obtain the cruising video of the target cruising pipeline, and the automatic cruising of the cruising pipeline is realized. And then, inputting each image in the patrol video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image, and determining whether a potential safety hazard exists in a target aviation oil pipeline at a corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and a preset region of interest, so that the intelligent analysis of the hidden danger of the aviation oil pipeline is realized, the efficiency of identifying the potential safety hazard of the aviation oil pipeline is improved, and the labor cost and the time cost are reduced.

As an embodiment, the hidden danger analysis model may be a target detection model based on YOLOv4 (You Only see Once), and the hidden danger analysis model includes an input module, a Backbone network (Backbone), a Neck network (tack), and an output module. The main Network adopts a CSPDarknet53 structure, and the neck Network includes an SPP (Spatial Pyramid Pooling) Network module, an FPN (Feature Pyramid Networks) structure, and a PAN (Path Aggregation Network) structure.

As shown in fig. 7, the hidden danger analysis model is trained in the following manner:

s701, a plurality of sample images are obtained. In this application embodiment, can patrol through the unmanned aerial vehicle field, gather the image data of patrolling of target aviation oil pipeline, regard the image frame in the image data of patrolling of gathering as the sample image.

S702, marking the hidden danger type and the hidden danger position of the target aviation oil pipeline in the sample image to obtain a sample data set. And marking the hidden danger category and hidden danger position of the target aviation oil pipeline in each sample image aiming at each obtained sample image, and constructing a sample data set.

For example, the types and positions of hidden dangers of target oil pipelines such as buildings, construction sites and construction machinery in each sample image can be labeled by using the existing labeling tool and using a rectangular frame according to each obtained sample image to obtain a labeled image, and each labeled image forms a sample data set.

Preferably, only images of the sample data set containing the hidden danger of the target aviation fuel pipeline are reserved, and images of the sample data set not containing the hidden danger of the target aviation fuel pipeline are deleted, so that the hidden danger analysis model can be trained accurately.

S703, determining training sample data from the sample data set, wherein the training sample data comprises a plurality of first sample images and the hidden danger categories and hidden danger positions of the first sample images.

After the sample data set is obtained, a part of sample data in the sample data set can be used as training sample data, and the other part of the sample data set can be used as test sample data. Optionally, the sample data set may be divided, and 80% of the sample data set may be used as training sample data, and the remaining 20% may be used as test sample data.

In order to distinguish training sample data from test sample data conveniently, sample images contained in the training sample data can be defined as the first sample images and the hidden danger categories and the hidden danger positions of the first sample images, and sample data contained in the test sample data can be defined as the second sample images and the hidden danger categories and the hidden danger positions of the second sample images.

S704, inputting each first sample image into an input module for image preprocessing to obtain a preprocessed image. The input module of the hidden danger analysis model can perform preprocessing on the input image, and the preprocessing can include normalization processing of the image size, data enhancement processing of the image and the like.

As an embodiment, the implementation process of inputting each first sample image into the input module for image preprocessing to obtain a preprocessed image may include:

inputting each first sample image into an input module to perform size normalization processing on each first sample image to obtain a normalized image; and performing data enhancement processing on each normalized image to obtain a preprocessed image.

And inputting each first sample image into an input module, so that the input module performs scaling processing on each first sample image, so as to scale the size of each first sample image to meet the size required by the hidden danger analysis model, realize size normalization of each first sample image, obtain a normalized image, and improve the training speed and precision of the hidden danger analysis model.

Furthermore, data enhancement processing is performed on each normalized image to increase the variability of each normalized image, so that the hidden danger analysis model can have higher robustness on sample images acquired under different environments. Preferably, the data enhancement process comprises: adjusting the brightness, chroma, saturation of the image and performing random scaling, cropping, flipping, rotating, etc., and may perform random erasing on the image, or CutOut (directly blocking the image) simulates blocking in the image, etc.

In one example, the normalized images may be expanded by a Mosaic data enhancement method, four different normalized images are spliced into one image by using Mosaic, and corresponding labels (i.e., hidden danger categories and hidden danger positions) are generated to improve the detection capability of the hidden danger analysis model on the small target. The Mosaic data enhancement method can carry out processing such as random scaling, random clipping and random arrangement on each normalized image, can enrich the background and small targets of the detected object, enables data of four images to be calculated once when Batch Normalization is calculated, and enables the mini-Batch size not to be large.

Preferably, the data can also be augmented using SAT (Self-countervailing Training). The generation and use of the SAT is divided into two stages, in the first stage, the hidden danger analysis model changes the original image (normalized image) instead of the model weight, and in this way, the hidden danger analysis model performs adversarial attack on itself, changes the original image, and deceives the model. In the second stage, training a hidden danger analysis model, detecting a target on the modified image in a normal mode, and improving the recognition capability of the hidden danger analysis model on the countermeasure sample.

In the embodiment of the application, each first sample image is input into the input module to be subjected to size normalization processing, so that the size normalization of each first sample image is realized, the training speed and precision of the hidden danger analysis model are further improved, data enhancement processing is performed on each normalized image, the variability of each normalized image is increased, and the hidden danger analysis model can have higher robustness and recognition capability on sample images obtained under different environments.

S705, inputting each preprocessed image into a backbone network for feature extraction, so as to output a first feature map of a first sample image corresponding to the preprocessed image.

The main Network adopts a CSPDarknet53 structure, the CSPDarknet53 structure combines Darknet53 with CSPNet (Cross Stage Partial Network), after combination, the CSPNet mainly works to split the stack of the residual block into a left part and a right part: the trunk part continues to stack the residual blocks, and the branch part is equivalent to a residual edge and is directly connected to the end through a small amount of processing.

Inputting each preprocessed image into a backbone network, and performing feature extraction on each preprocessed image by using the CSPDarknet53 to output a first feature map of a first sample image corresponding to the preprocessed image. CSPDarknet53 contains 29 convolutional layers and has enough depth and receptive field, 5 CSP modules are added to the CSPDarknet53 on the basis of Darknet53, the CSP divides the feature mapping of a basic layer (input module) into two parts for processing, so that gradient flow is propagated through different network paths to reduce the repeated calculation of the gradient, and then the gradient flow is combined through a cross-layer splicing structure, thereby reducing the calculation amount while ensuring the model accuracy. The smooth Mish is used as the activation function, the hard zero boundary of the ReLU serving as the activation function is avoided, more information is allowed to enter the network, and therefore the accuracy and the generalization capability of the model are improved. Dropblock (block discarding, a regularization method) is used in the model in an alternating mode, the characteristics of adjacent regions are discarded, the defect that Dropout does not have obvious effect on the convolution layer is overcome, overfitting is prevented beneficially, and the generalization capability of the model is improved. Illustratively, the dimensions of the input preprocessed image of the CSPDarknet53 structure are 608 × 606 × 3.

S706, inputting each first feature map into the neck network for feature extraction, so as to output a second feature map of the first sample image.

The neck network further extracts features of each first feature map from the backbone network to output a second feature map of the first sample image.

In one example, the categories of potential hazards include: building category, construction site category, and construction machine category, and accordingly, the second feature map may be tensors of three different scales for identifying a small target, a medium target, and a large target, respectively.

The hidden danger analysis model can be a target detection model based on YOLOv4, and the idea of a prediction box in YOLOv4 is as follows: the feature maps (feature maps) output by the upper layer are processed by utilizing maximum pooling of four different scales respectively. The sizes of the maximum pooled pooling cores of four different scales are respectively 13x13, 9x9, 5x5 and 1x1, wherein 1x1 is equivalent to no processing, so that the network is not limited by a fixed input size.

The neck network includes an SPP network module, an FPN structure, and a PAN structure. The SPP performs maximum pooling by using pooling kernels with the sizes of 1 × 1, 5 × 5, 9 × 9 and 13 × 13, and then splices the results together to perform multi-scale fusion on the features. The FPN enlarges the original feature map layer by layer from top to bottom through deconvolution to construct a feature pyramid, so that semantic features of objects with different sizes extracted by a network are facilitated, and a model can identify the same object with different sizes and scales. PAN has used for reference image segmentation field PANet algorithm, from bottom to top extraction characteristic to the unmanned aerial vehicle through the concatenation is cruising the pipeline hidden danger analytical method and is carrying out the polymerization to the different layer parameter that comes from FPN, and then strengthens the positional capability of network.

And S707, inputting the second feature maps into the output module, and calculating the category and the position of each second feature map to obtain the predicted potential hazard category and the predicted potential hazard position of the first sample image.

In one example, the output module outputs bounding boxes (locations) and corresponding categories of the identified hidden danger for three different scales of the second feature map. The output module calculates the category and the position of each second feature map so as to realize multi-scale prediction of the first sample image and improve the detection performance of the hidden danger analysis model on different scale targets.

In one example, the process of calculating the category by the output module by using the extracted feature map comprises the following steps: small, medium and large targets are detected by 3 different scales respectively. Three detections are performed in the output module network, namely 32 times down-sampling, 16 times down-sampling and 8 times down-sampling detection. The reason for using up-sample in the output module network is that the deeper the network, the better the feature expression effect, for example, in 16 times down-sampling detection, if the feature of the fourth down-sampling is directly used for detection, the shallow feature is used, and the effect is not good. If we want to use 32 times down-sampled features, but the size of the deep features is too small, so up-sample with step size of 2 is used, and the feature map obtained by 32 times down-sampling is increased by one time, and becomes 16 times down-sampling. Similarly, 8-fold downsampling also performs upsampling with step size of 2 on 16-fold downsampled features, so that deep features can be used for prediction. The 16-fold downsampling and the 8-fold downsampling use deep features well by means of upsampling, but the shallow feature maps obtained by performing 4-fold downsampling and 3-fold downsampling are the same in size.

And S708, calculating the current loss according to the predicted hidden danger category and the predicted hidden danger position of the first sample image and the hidden danger category and the hidden danger position of the first sample image.

And S709, adjusting the training parameters of the hidden danger analysis model according to the current loss until a preset ending condition is met, and obtaining the trained hidden danger analysis model. The preset ending condition may be set according to an actual requirement, for example, the preset ending condition may be that the current loss reaches the set precision, or that the training of the hidden danger analysis model reaches the set iteration number, and the like.

Illustratively, when the hidden danger analysis model is trained, the size of an input image (a first sample image) may be 608 × 608, the batch size is 8, the initial learning rate is 0.001, an exponential decay strategy is used, the training round is 50, and then the trained hidden danger analysis model is trained by using the training sample data to obtain the trained hidden danger analysis model.

In the embodiment of the application, the hidden danger analysis model is a target detection model based on YOLOv4, a CSPDarknet53 structure is adopted in a main network of the hidden danger analysis model, a neck network comprises an SPP network module, an FPN structure and a PAN structure, characteristics of a sample image can be accurately extracted, accurate detection of the hidden danger of the target aviation oil pipeline can be realized, the model detection speed is high, and intelligent identification and analysis of the hidden danger of the target aviation oil pipeline are realized.

As an embodiment, the current loss of the hidden danger analysis model may be determined according to an intersection ratio of a hidden danger prediction frame and a hidden danger real frame of the first sample image, a center point distance, and an aspect ratio.

In one example, the hazard analysis model uses the CIOU loss function to measure the difference between the predicted box and the true box. Specifically, the minimum closure area of the prediction frame and the real frame (that is, the area of the minimum frame including both the prediction frame and the real frame) is calculated by an IOU (Intersection over Union), samples below the IOU threshold are removed, then a final prediction frame is obtained by an NMS (Non-Maximum Suppression) algorithm, a loss value is calculated, after each iteration is finished, a verification set is transmitted to the network, the reliability of the model is verified, and the loss value and the accuracy are calculated.

For example, the current loss of the hidden danger analysis model can be expressed as:

wherein, CIOU _loss Representing the current loss, the IOU represents the intersection ratio of the hidden danger prediction frame of the first sample image and the hidden danger real frame of the first sample image, b and b ^gt Respectively representing the central points, rho, of the hidden danger prediction frame and the hidden danger real frame ² (b,b ^gt ) Representing the Euclidean distance between the central points of the hidden danger prediction frame and the hidden danger real frame, c representing the diagonal length of the minimum rectangular frame covering the hidden danger prediction frame and the hidden danger real frame, alpha representing a weight coefficient, v representing the similarity value of the length-width ratio, h and h ^gt Respectively representing the heights, omega and omega, of the hidden danger prediction frame and the hidden danger real frame ^gt And respectively representing the widths of the hidden danger prediction frame and the hidden danger real frame.

Using YOLOv4 network to perform target recognition on images, each image generates 22743 prediction frames, each prediction frame contains 5+ n values, which are the center point coordinate (x, y), width and height (w, h), confidence c, and the probability of n categories, and accordingly, the hidden danger prediction frame includes: center point coordinates, height values, width values, confidence values, and probabilities of the categories. According to the value of the confidence coefficient c, some prediction boxes with low confidence coefficients can be filtered, for example, in the embodiment of the present application, the confidence coefficient threshold of the hidden danger analysis model can be set to 0.5, CIOU-NMS screening is performed on the remaining prediction boxes to screen out too many repeated prediction boxes, and the remaining prediction boxes are used as the final detection result.

In the embodiment of the application, the current loss of the hidden danger analysis model is calculated by using the CIOU loss function, and the coverage area, the central point distance and the length-width ratio of each frame of the prediction frame and the real frame are considered, so that the detection effect of the hidden danger analysis model can be better detected, and the training speed of the hidden danger analysis model is improved.

As an embodiment, in the training process of the hidden danger analysis model, after the step S702 labels the hidden danger category and the hidden danger position of the target aviation fuel pipeline in the sample image and obtains the sample data set, the method may further include:

determining test sample data from a sample data set, wherein the test sample data comprises a plurality of second sample images and hidden danger categories and hidden danger positions of the second sample images;

inputting each second sample image into the trained hidden danger analysis model to output the predicted hidden danger category and the predicted hidden danger position of the second sample image;

and thirdly, calculating the omission factor and the false alarm factor of the hidden danger analysis model according to the predicted hidden danger category and the predicted hidden danger position of the second sample image and the hidden danger category and the hidden danger position of the second sample image.

The method comprises the steps of marking the hidden danger category and the hidden danger position of a target oil pipeline in a sample image to obtain a sample data set, determining test sample data from the sample data set, inputting a second sample image in the test sample data into an input module for image preprocessing to obtain preprocessed images, inputting the preprocessed images into a backbone network for feature extraction to output a first feature map of a second sample image corresponding to the preprocessed images, inputting the first feature maps into a neck network for feature extraction to output a second feature map of the second sample image, inputting the second feature maps into an output module to calculate the category and the position of the second feature maps to obtain the predicted hidden danger category and the predicted hidden danger position of the second sample image.

And respectively calculating the predicted hidden danger category and the predicted hidden danger position of the second sample image and the difference between the hidden danger category and the hidden danger position of the second sample image to obtain the omission ratio and the false alarm ratio of the hidden danger analysis model.

In the training process of the hidden danger analysis model, the steps S703 to S708 may be performed synchronously with the steps one to three, or may not be performed synchronously, which is not limited in the embodiment of the present application.

In step S709, the training parameters of the hidden danger analysis model are adjusted according to the current loss until a preset termination condition is satisfied, so as to obtain a trained hidden danger analysis model, including:

After the current loss is obtained through calculation, the training parameters of the hidden danger analysis model can be adjusted according to the current loss and the missing rate and the false alarm rate of the hidden danger analysis model until the preset ending condition is met, and the training of the hidden danger analysis model is completed under the condition that the missing rate and the false alarm rate of the hidden danger analysis model meet the preset conditions. The hidden danger analysis model comprises a hidden danger analysis model and a hidden danger analysis model, wherein the missing rate and the false alarm rate of the hidden danger analysis model meet preset conditions, and the missing rate and the false alarm rate of the hidden danger analysis model are smaller than set thresholds. For example, the set threshold corresponding to the missing detection rate may be set to 15%, the set threshold corresponding to the false alarm rate may be set to 20%, and so on.

In the embodiment of the application, in the training process of the hidden danger analysis model, the hidden danger analysis model is trained through training sample data, the hidden danger analysis model is tested according to the test sample data, and when the loss of the hidden danger analysis model and the missing rate and the false rate meet the set conditions, the training of the hidden danger analysis model is completed, so that the trained hidden danger analysis model can be used for more accurately detecting the hidden danger of the target aviation fuel pipeline.

clustering a labeling frame corresponding to the hidden danger position based on the hidden danger position of a target aviation fuel pipeline labeled in each labeled sample image in the sample data set to obtain a clustering result corresponding to the hidden danger position; and updating the position of the hidden danger according to the clustering result.

The hidden danger analysis model is a target detection model based on YOLOv4, 9 detection frames with different sizes and aspect ratios are designed in the YOLOv4, and then corresponding detection frames can be set according to the sizes of hidden dangers (buildings, construction sites and construction machinery) of different aviation oil pipelines.

In one example, for each labeled sample image in the sample data set, based on the hidden danger position of the labeled target aviation oil pipeline in the labeled sample image, a K-means (K-means) clustering algorithm may be adopted to cluster the size of the labeling box (i.e., the detection box) corresponding to the hidden danger position to obtain a clustering result corresponding to the hidden danger position, and then the hidden danger position is updated according to the clustering result.

In the embodiment of the application, for each labeled sample image in a sample data set, based on the hidden danger position of the target aviation oil pipeline labeled in the labeled sample image, clustering is performed on the labeling frames corresponding to the hidden danger positions to obtain the clustering result corresponding to the hidden danger positions, and the hidden danger positions are further updated according to the clustering result, so that the hidden danger analysis model can more accurately identify the hidden danger of the target aviation oil pipeline.

Based on the same application concept as the method, the embodiment of the present application further provides an unmanned aerial vehicle cruise pipeline hidden danger analysis device, as shown in fig. 8, the device may include:

the information acquisition unit 801 is used for acquiring pre-generated route information, wherein the route information comprises routes of the unmanned aerial vehicle and flight parameter information; the route comprises a plurality of waypoints, and the waypoints are determined based on a three-dimensional point cloud model of a target navigation oil pipeline inspection area;

the route patrol unit 802 is used for performing automatic patrol by using the unmanned aerial vehicle according to the route information to obtain a patrol video of the target aviation oil pipeline;

a hidden danger analysis unit 803, configured to input each image in the inspection video into the hidden danger analysis model one by one, so as to output a hidden danger analysis result of each image; the hidden danger analysis model is obtained by training a hidden danger result of a sample image and the sample image, the hidden danger analysis result comprises a hidden danger category and a hidden danger position, and the hidden danger category comprises: a building category, a construction site category, and a construction machine category;

and the hidden danger determining unit 804 is configured to determine whether a potential safety hazard exists in the target aviation fuel pipeline at the corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and the preset region of interest.

The utility model provides a pair of unmanned aerial vehicle pipeline hidden danger analytical equipment that cruises, the route information that can acquire the route and the flight parameter information including unmanned aerial vehicle of pre-generation, a plurality of waypoints in the route are based on the regional three-dimensional point cloud model of target aviation oil pipeline inspection is confirmed, utilize unmanned aerial vehicle to carry out automatic inspection according to the route information, obtain the inspection video of target aviation oil pipeline, realized the automatic inspection of aviation oil pipeline. And then, inputting each image in the patrol video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image, and determining whether a potential safety hazard exists in a target aviation oil pipeline at a corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and a preset region of interest, so that the intelligent analysis of the hidden danger of the aviation oil pipeline is realized, the efficiency of identifying the potential safety hazard of the aviation oil pipeline is improved, and the labor cost and the time cost are reduced.

As an embodiment, the apparatus further comprises: the area determining unit is used for determining an area in a preset range around the target aviation oil pipeline as a patrol area; the point cloud data acquisition unit is used for acquiring point cloud data of the inspection area to obtain target point cloud data of the target oil and gas pipeline; the model generation unit is used for generating a three-dimensional point cloud model of the target aviation oil pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of target ground objects in a pre-collected inspection area; the navigation point determining unit is used for determining a plurality of navigation points based on points with the height larger than a preset threshold value of ground object coordinates in the three-dimensional point cloud model; the route generation unit is used for connecting a plurality of waypoints in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline; the system comprises a route information generating unit, a route information generating unit and a route information generating unit, wherein the route information generating unit is used for setting flight parameter information for a route to obtain route information; the flight parameter information includes at least one of a flight speed, a photographing distance, and a photographing attitude.

As an embodiment, the hidden danger analysis model is a target detection model based on YOLOv4, and includes an input module, a backbone network, a neck network and an output module, where the backbone network adopts a CSPDarknet53 structure, and the neck network includes a spatial pyramid pooling SPP network module, a feature pyramid network FPN structure and a path aggregation network PAN structure; the above-mentioned device still includes:

an image acquisition unit configured to acquire a plurality of sample images; the image labeling unit is used for labeling the hidden danger category and the hidden danger position of the target aviation oil pipeline in the sample image to obtain a sample data set; the training sample determining unit is used for determining training sample data from the sample data set, wherein the training sample data comprise a plurality of first sample images and hidden danger categories and hidden danger positions of the first sample images; the image processing unit is used for inputting each first sample image into the input module for image preprocessing so as to obtain a preprocessed image; the first feature extraction unit is used for inputting each preprocessed image into a backbone network for feature extraction so as to output a first feature map of a first sample image corresponding to the preprocessed image; the second characteristic extraction unit is used for inputting each first characteristic diagram into the neck network to carry out characteristic extraction so as to output a second characteristic diagram of the first sample image; the first hidden danger prediction unit is used for inputting each second feature map into the output module to calculate the category and the position of each second feature map so as to obtain the predicted hidden danger category and the predicted hidden danger position of the first sample image; the loss calculation unit is used for calculating the current loss according to the predicted hidden danger type and the predicted hidden danger position of the first sample image and the hidden danger type and the hidden danger position of the first sample image; and the model training unit is used for adjusting the training parameters of the hidden danger analysis model according to the current loss until a preset ending condition is met, so as to obtain the trained hidden danger analysis model.

As an embodiment, the image processing unit is specifically configured to: inputting each first sample image into an input module to perform size normalization processing on each first sample image to obtain a normalized image; and performing data enhancement processing on each normalized image to obtain a preprocessed image.

As an embodiment, the above apparatus further comprises: the test sample determining unit is used for determining test sample data from the sample data set, wherein the test sample data comprises a plurality of second sample images and the hidden danger types and hidden danger positions of the second sample images; the second hidden danger prediction unit is used for inputting each second sample image into the trained hidden danger analysis model so as to output the predicted hidden danger category and the predicted hidden danger position of the second sample image; the accuracy calculation unit is used for calculating the omission factor and the false alarm rate of the hidden danger analysis model according to the predicted hidden danger type and the predicted hidden danger position of the second sample image and the hidden danger type and the hidden danger position of the second sample image; the model training unit is specifically configured to: and adjusting the training parameters of the hidden danger analysis model according to the current loss and the missing detection rate and the false alarm rate of the hidden danger analysis model until the preset ending condition is met, and obtaining the trained hidden danger analysis model when the missing detection rate and the false alarm rate of the hidden danger analysis model meet the preset conditions.

As an embodiment, the above apparatus further comprises: the clustering unit is used for clustering a labeling frame corresponding to a hidden danger position according to a hidden danger position of a target aviation fuel pipeline labeled in each labeled sample image in the sample data set to obtain a clustering result corresponding to the hidden danger position; and the updating unit is used for updating the position of the hidden danger according to the clustering result.

As an embodiment, the current loss of the hidden danger analysis model is determined according to an intersection ratio of a hidden danger prediction frame and a hidden danger real frame of the first sample image, a center point distance, and an aspect ratio.

An embodiment of this application still provides an unmanned aerial vehicle pipeline hidden danger analytic system that cruises, refer to fig. 9, and unmanned aerial vehicle pipeline hidden danger analytic system that cruises includes: an airline planning module 901, an airline patrol module 902 and a hidden danger analysis platform 903;

the route planning module 901 is configured to determine an area within a preset range around a target aviation oil pipeline as a patrol area; collecting point cloud data of the inspection area to obtain target point cloud data of a target aviation oil pipeline; generating a three-dimensional point cloud model of the target oil pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of target ground objects in a pre-collected inspection area; determining a plurality of navigation points based on points with the height greater than a preset threshold value of the ground object coordinate in the three-dimensional point cloud model; connecting a plurality of navigation points in the three-dimensional point cloud model to generate a route corresponding to a target oil pipeline; setting flight parameter information for the air route to obtain air route information; the flight parameter information comprises at least one item of flight speed, shooting distance and shooting attitude;

the route patrol module 902 is used for performing automatic patrol by using the unmanned aerial vehicle according to route information to obtain a patrol video of the target aviation oil pipeline, and sending the patrol video to the hidden danger analysis platform 903;

the hidden danger analysis platform 903 is used for inputting each image in the patrol video into the hidden danger analysis model one by one so as to output a hidden danger analysis result of each image; determining whether potential safety hazards exist in the target aviation fuel pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and a preset region of interest, and generating a potential safety hazard analysis report under the condition that the potential safety hazards exist in the target aviation fuel pipeline; the hidden danger analysis model is obtained by training hidden danger results of sample images, the hidden danger analysis results comprise hidden danger categories and hidden danger positions, and the hidden danger categories comprise: a building category, a construction site category, and a construction machine category.

In one example, after the hidden danger analysis model is trained, the trained hidden danger analysis model is deployed on a GPU (graphics processing unit) server to be applied to a fuel oil pipeline hidden danger identification and analysis system.

The route patrol module 902 performs automatic patrol by using the unmanned aerial vehicle according to the route information generated by the route planning module 901, obtains a patrol video of the target oil and gas pipeline, and sends the patrol video to the hidden danger analysis platform 903. The hidden danger analysis platform 903 (GPU server) performs hidden danger analysis on each frame of image in the patrol video by using the trained hidden danger analysis model deployed on the GPU server to obtain a hidden danger analysis result (including a hidden danger type and a hidden danger position, wherein the hidden danger type comprises a building type, a construction site type and a construction machinery type) of each image, determines whether a target aviation oil pipeline at a position corresponding to each image has a potential safety hazard or not according to the hidden danger position in each image hidden danger analysis result and a preset region of interest, and generates a hidden danger analysis report according to the image with the potential safety hazard if the target aviation oil pipeline has the potential safety hazard.

The unmanned aerial vehicle cruising pipeline hidden danger analysis system provided by the embodiment of the application acquires point cloud data of a cruising area of a target aviation oil pipeline to obtain target point cloud data of the target aviation oil pipeline, generates a three-dimensional point cloud model of the target aviation oil pipeline based on the target point cloud data and preset control point coordinates, further determines a plurality of navigation points with different heights based on points with the heights larger than a preset threshold value of a ground object coordinate in the three-dimensional point cloud model, connects the navigation points in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline, sets flight parameter information for the route to obtain route information, realizes automatic planning of the unmanned aerial vehicle cruising pipeline, and enables the determined route to change along with terrain because the navigation points are determined based on the points with the heights larger than the preset threshold value of the ground object coordinate in the three-dimensional point cloud model, thereby reducing the problem that the cruising at a lower place cannot be clearly caused by setting a fixed flight height for avoiding the high points.

And the unmanned aerial vehicle is used for automatically patrolling according to the route information to obtain a patrol video of the target aviation oil pipeline, so that the automatic patrol of the aviation oil pipeline is realized. And then, inputting each image in the patrol video into the hidden danger analysis model one by one to output a hidden danger analysis result of each image, and determining whether a potential safety hazard exists in a target aviation oil pipeline at a corresponding position of each image based on the hidden danger position in each image hidden danger analysis result and a preset region of interest, so that the intelligent analysis of the hidden danger of the aviation oil pipeline is realized, the efficiency of identifying the potential safety hazard of the aviation oil pipeline is improved, and the labor cost and the time cost are reduced.

Fig. 10 is a block diagram of an electronic device 100 according to an embodiment of the present application. The electronic device 100 includes one or more processors 110 for implementing the unmanned aerial vehicle cruise pipeline hazard analysis method as described above.

In some embodiments, electronic device 100 may include computer-readable storage medium 140, and computer-readable storage medium 140 may store a program that may be invoked by processor 110, and may include non-volatile storage media. In some embodiments, electronic device 100 may include memory 130 and interface 120. In some embodiments, electronic device 100 may also include other hardware depending on the application.

The computer-readable storage medium 140 of the embodiment of the present application stores thereon a program, and the program is used for implementing the unmanned aerial vehicle cruise pipeline hidden danger analysis method described above when being executed by the processor 110.

This application may take the form of a computer program product that is embodied on one or more computer-readable storage media 140 (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-readable storage media 140 includes both permanent and non-permanent, removable and non-removable media, and may implement information storage in any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer-readable storage media 140 include, but are not limited to: phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by a computing device.

The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the phrases "comprising a" \8230; "defining an element do not exclude the presence of additional like elements in the process, method, article, or apparatus that comprises the element.

Claims

1. An unmanned aerial vehicle cruising pipeline hidden danger analysis method is characterized by comprising the following steps:

acquiring pre-generated route information, wherein the route information comprises a route of the unmanned aerial vehicle and flight parameter information; the flight path comprises a plurality of flight points, and the flight points are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area;

and determining whether the potential safety hazard exists in the target aviation fuel pipeline at the corresponding position of each image or not based on the potential safety hazard position in each image potential safety hazard analysis result and the preset region of interest.

2. The unmanned aerial vehicle cruise pipeline potential hazard analysis method according to claim 1, further comprising:

determining an area in a preset range around a target aviation oil pipeline as a patrol area;

generating a three-dimensional point cloud model of the target aviation oil pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of target ground objects in the inspection area, which are acquired in advance;

3. The unmanned aerial vehicle cruise pipeline potential hazard analysis method according to claim 1, wherein the potential hazard analysis model is a YOLOv 4-based target detection model, the potential hazard analysis model comprises an input module, a backbone network, a neck network and an output module, the backbone network adopts a CSPDarknet53 structure, and the neck network comprises a spatial pyramid pooling SPP network module, a feature pyramid network FPN structure and a path aggregation network PAN structure; training the hidden danger analysis model in the following way:

acquiring a plurality of sample images;

marking the hidden danger type and the hidden danger position of the target aviation oil pipeline in the sample image to obtain a sample data set;

inputting each preprocessed image into the backbone network for feature extraction to output a first feature map of a first sample image corresponding to the preprocessed image;

inputting each second feature map into the output module, and calculating the category and the position of each second feature map to obtain the category and the position of the predicted hidden danger of the first sample image;

4. The unmanned aerial vehicle cruise pipeline hidden danger analysis method according to claim 3, wherein the inputting each first sample image into the input module for image preprocessing to obtain a preprocessed image comprises:

5. The unmanned aerial vehicle cruise pipeline hidden danger analysis method according to claim 3, wherein after the hidden danger category and the hidden danger position of the target aviation oil pipeline in the sample image are labeled to obtain a sample data set, the method further comprises:

6. The unmanned aerial vehicle cruise pipeline potential hazard analysis method according to any one of claims 3 to 5, wherein after the potential hazard category and the potential hazard position of the target aviation oil pipeline in the sample image are labeled to obtain a sample data set, the method further comprises:

7. The unmanned aerial vehicle cruise pipeline hidden danger analysis method as claimed in any one of claims 3 to 5, wherein the current loss of the hidden danger analysis model is determined according to the intersection ratio of a hidden danger prediction frame and a hidden danger real frame of the first sample image, the central point distance and the aspect ratio.

8. The utility model provides an unmanned aerial vehicle pipeline hidden danger analytical equipment that cruises which characterized in that, the device includes:

the information acquisition unit is used for acquiring pre-generated route information, wherein the route information comprises a route of the unmanned aerial vehicle and flight parameter information; the flight line comprises a plurality of flight points, and the flight points are determined based on a three-dimensional point cloud model of a target oil pipeline inspection area;

the route patrol unit is used for automatically patrolling by using an unmanned aerial vehicle according to the route information to obtain a patrol video of the target oil pipeline;

9. The utility model provides an unmanned aerial vehicle pipeline hidden danger analytic system that cruises, its characterized in that, the system includes: the system comprises a route planning module, a route inspection module and a hidden danger analysis platform;

the route planning module is used for determining an area in a preset range around the target aviation oil pipeline as a patrol area; collecting point cloud data of the inspection area to obtain target point cloud data of the target aviation oil pipeline; generating a three-dimensional point cloud model of the target aviation oil pipeline based on the target point cloud data and preset control point coordinates; the control point coordinates are real coordinates of target ground objects in the inspection area, which are acquired in advance; determining a plurality of navigation points based on points with the height greater than a preset threshold value of the ground object coordinate in the three-dimensional point cloud model; connecting the multiple navigation points in the three-dimensional point cloud model to generate a route corresponding to the target aviation oil pipeline; setting flight parameter information for the air route to obtain air route information; the flight parameter information comprises at least one of flight speed, shooting distance and shooting attitude;

10. An electronic device comprising a processor and a memory;

a memory for storing a computer program;

the processor is used for realizing the unmanned aerial vehicle cruising pipeline hidden danger analysis method as claimed in any one of claims 1 to 7 when executing the program stored in the memory.