CN112215805A - Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance - Google Patents
Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance Download PDFInfo
- Publication number
- CN112215805A CN112215805A CN202011000233.3A CN202011000233A CN112215805A CN 112215805 A CN112215805 A CN 112215805A CN 202011000233 A CN202011000233 A CN 202011000233A CN 112215805 A CN112215805 A CN 112215805A
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- inspection
- data
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 105
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012423 maintenance Methods 0.000 title claims abstract description 16
- 201000010099 disease Diseases 0.000 claims abstract description 41
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 41
- 238000010191 image analysis Methods 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims description 27
- 238000001514 detection method Methods 0.000 claims description 26
- 238000013523 data management Methods 0.000 claims description 16
- 238000003709 image segmentation Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 6
- 230000008030 elimination Effects 0.000 claims description 6
- 238000003379 elimination reaction Methods 0.000 claims description 6
- 238000007726 management method Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 description 7
- 230000001133 acceleration Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000003708 edge detection Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- DMBHHRLKUKUOEG-UHFFFAOYSA-N diphenylamine Chemical compound C=1C=CC=CC=1NC1=CC=CC=C1 DMBHHRLKUKUOEG-UHFFFAOYSA-N 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000009991 scouring Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
- G07C1/20—Checking timed patrols, e.g. of watchman
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30184—Infrastructure
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method and a system for inspecting a highway bridge slope maintenance unmanned aerial vehicle, wherein the method comprises the following steps: acquiring polling task data of the unmanned aerial vehicle; controlling an unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data; receiving video data uploaded by an unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection; and carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part. This application can reduce when patrolling and examining personnel work load and work danger coefficient, can also widen to a certain extent and patrol and examine the position, increase and patrol and examine the number of times to reduce the risk that the bridge produced the potential safety hazard. The method and the device can be widely applied to the field of image processing.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method and a system for inspecting a highway bridge slope maintenance unmanned aerial vehicle.
Background
Road administration patrol is an effective management means for protecting road safety, serving public trips and strengthening team self construction. By inspection, the road condition is comprehensively known in real time so as to find and process problems in time.
Unmanned Aerial vehicle (uav) refers to an aircraft that does not carry an operator and can fly autonomously or be remotely piloted, and is an important vehicle for artificial intelligence.
At present, the traditional bridge inspection method is to observe by human eyes and inspect each component of the bridge for diseases by using auxiliary tools such as telescopes, supports, ships, ascending vehicles and/or bridge inspection vehicles. The current inspection methods have limitations: firstly, the bridge structure covered by the inspection position has small area; secondly, the frequency of the checking times is less. Therefore, the risk of potential safety hazards of the bridge is increased.
Disclosure of Invention
In order to solve one of the above technical problems to some extent, the present invention aims to: the utility model provides a highway bridge side slope maintenance unmanned aerial vehicle patrols and examines method and system, and it can widen the position of patrolling and examining, reduces the bridge and produces the risk of potential safety hazard.
In a first aspect, an embodiment of the present invention provides:
an unmanned aerial vehicle inspection method for highway bridge slope maintenance comprises the following steps:
acquiring polling task data of the unmanned aerial vehicle;
controlling an unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data;
receiving video data uploaded by an unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection;
and carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part.
Further, the image analysis of the video data specifically includes:
performing image analysis on the video data through a pre-trained target detection network;
the training process of the target detection network comprises the following steps:
constructing training sample data, wherein the training sample data comprises a plurality of training images marked with first feature points in advance;
constructing a target detection network to be trained;
and training the target detection network to be trained through the training sample data.
Further, the performing image analysis on the video data through a pre-trained target detection network includes:
carrying out target detection on a plurality of images in the video data through a pre-trained target detection network, and extracting second feature points in the plurality of images;
and analyzing the disease state of the inspection part according to the second characteristic point.
Further, the performing image analysis on the video data includes:
carrying out image segmentation on the plurality of images to obtain a target image;
extracting the features of the target image, and extracting a third feature point of the target image;
and analyzing the disease state of the highway bridge slope according to the third characteristic point.
Further, before the step of image segmentation of the images, the method further comprises the following steps:
respectively carrying out image enhancement on the plurality of images;
shadow elimination is carried out on a plurality of images after image enhancement processing;
and filtering out noise of a plurality of images after shadow elimination.
Further, according to patrol and examine task data control unmanned aerial vehicle and carry out public road bridge roof beam side slope and patrol and examine, include:
controlling the flight path of the unmanned aerial vehicle according to the routing inspection path data in the routing inspection task data;
acquiring real-time position data of the unmanned aerial vehicle during flight;
calculating the linear distance between the real-time position data and target position data in the routing inspection task data;
after the linear distance is determined to belong to the preset range, the flight speed of the unmanned aerial vehicle and the shooting angle and the shooting mode of a camera on the unmanned aerial vehicle are adjusted, and video data are acquired for the target position corresponding to the current target position data.
In a second aspect, an embodiment of the present invention provides:
the utility model provides a highway bridge side slope maintenance unmanned aerial vehicle system of patrolling and examining, includes removes end operating system, unmanned aerial vehicle, data management platform and server, remove end operating system, unmanned aerial vehicle and data management platform all with the server communication, the server is used for carrying out following step:
acquiring polling task data of the unmanned aerial vehicle;
controlling an unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data;
receiving video data uploaded by an unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection;
and carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part.
Further, the data management platform is used for displaying the inspection part and the disease state corresponding to the inspection part through the three-dimensional model.
Further, the data management platform is also used for constructing a disease state library of the highway bridge side slope and carrying out time sequence management on the disease state.
Further, be equipped with orientation module and shock attenuation module on the unmanned aerial vehicle, orientation module includes IMU module, GNSS module, light stream sensor module, infrared sensor module and RTK module.
The embodiment of the invention has the beneficial effects that: according to the embodiment of the invention, the patrol inspection task data of the unmanned aerial vehicle is obtained, the unmanned aerial vehicle is controlled to patrol the highway bridge slope according to the patrol inspection task data, then the video data consisting of a plurality of images acquired by the unmanned aerial vehicle when the unmanned aerial vehicle patrols the highway bridge slope is received, and after the video data is subjected to image processing, a patrol inspection report containing the patrol inspection part and the disease state corresponding to the patrol inspection part is generated, so that the patrol inspection position can be widened to a certain extent while the workload and the working risk coefficient of patrol inspection personnel are reduced, and the patrol inspection frequency is increased, so that the risk of potential safety hazard of the bridge is reduced.
Drawings
Fig. 1 is a flowchart of a method for routing inspection by a highway bridge slope maintenance unmanned aerial vehicle according to an embodiment of the invention;
fig. 2 is a block diagram of a road bridge slope maintenance unmanned aerial vehicle inspection system according to a specific embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.
First, terms appearing in the present application are explained:
an IMU: an Inertial Measurement Unit, a device for measuring the triangular attitude angle and acceleration of an object.
GNSS: global Navigation Satellite System, Global Navigation Satellite System.
RTK: real-time kinematic is abbreviated as Real-time kinematic. The real-time dynamic carrier phase differential technology is a differential technology for processing the observed quantity of carrier phases of two measuring stations in real time, and the carrier phases acquired by a reference station are sent to a user receiver to solve and calculate coordinates. Which is a commonly used satellite positioning measurement method.
Referring to fig. 1, an embodiment of the present invention provides a method for inspecting a highway bridge slope maintenance unmanned aerial vehicle, and the embodiment is applied to a server of the system shown in fig. 2, where the server is respectively in communication with a mobile terminal operating system, the unmanned aerial vehicle, and a data management platform. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and a big data and artificial intelligence platform.
The present embodiment includes steps S110 to S140:
s110, acquiring inspection task data of the unmanned aerial vehicle; the unmanned aerial vehicle patrol task data comprise position information of a road, a bridge and/or a side slope which need to be patrolled in a patrol process, route information of the position, and information which needs to be operated at the position. In some embodiments, the patrol task data may be patrol data obtained in a database of the server after receiving an execution instruction uploaded by the mobile terminal operating system.
In other embodiments, the patrol personnel selects specific patrol items on a human-computer interaction interface of the mobile terminal operating system, for example, specific positions of a road, a bridge and/or a slope where the current unmanned aerial vehicle needs to patrol, specific information which needs to be collected on the road, the bridge and/or the slope, and the like, and after receiving the options uploaded by the mobile terminal operating system, the server generates patrol task data consisting of a flight route, a flight speed and operation information on the specific positions according to the uploaded options.
S120, controlling the unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data; the unmanned aerial vehicle control system comprises an unmanned aerial vehicle, a camera and a controller, wherein the unmanned aerial vehicle is controlled to fly to a pier A position, the unmanned aerial vehicle is controlled to hover at the pier A position, in the hovering process, the attitude angle of the unmanned aerial vehicle and the attitude angle of the camera on the unmanned aerial vehicle are adjusted, after the attitude angle is adjusted, the camera is controlled to perform video recording or photographing and other operations, after the current position is determined to complete the video recording or photographing and other operations, the unmanned aerial vehicle is controlled to fly to the next position, and the unmanned aerial vehicle is controlled to return to the ground until all positions needing to acquire videos or images are completed.
In some embodiments, the executing process of step S120 specifically includes:
and controlling the flight path of the unmanned aerial vehicle according to the routing inspection path data in the routing inspection task data so as to avoid the flight path deviation of the unmanned aerial vehicle. For example, during flight, when an obstacle is encountered, the flight angle is appropriately adjusted to control the drone to pass through the obstacle, and after passing through the obstacle, the drone is controlled to return to the head rail.
Acquiring real-time position data of the unmanned aerial vehicle during flight; the space position coordinate when acquiring unmanned aerial vehicle flight specifically, this space position coordinate can be relative predetermine the space position coordinate of benchmark.
Calculating the linear distance between the real-time position data and target position data in the routing inspection task data; the linear distance is the linear distance of two spatial points, which changes in real time because the real time position data changes.
After the linear distance is determined to belong to the preset range, the flight speed of the unmanned aerial vehicle and the shooting angle and the shooting mode of a camera on the unmanned aerial vehicle are adjusted, and video data are acquired for the target position corresponding to the current target position data. The preset range is a preset distance range. When unmanned aerial vehicle's real-time straight line distance is in this distance range, it can effectively gather the video data or the image of current position at the current position to show unmanned aerial vehicle.
The embodiment controls the video data acquisition process of the unmanned aerial vehicle through the linear distance between the real-time position of the unmanned aerial vehicle and the target position, so that the accuracy of data acquisition of the unmanned aerial vehicle is improved.
S130, receiving video data uploaded by the unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection; in addition, the unmanned aerial vehicle sends the surrounding video data on the flight route to the server, so that the server adjusts the flight route of the unmanned aerial vehicle in the subsequent processing process. In some embodiments, the data on the drone may also be forwarded to a server through a mobile-side operating system.
S140, carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part. The inspection part comprises a bridge tower, a main cable, a cable clamp, a sling, a stay cable, an anchorage, an expansion joint, a support, a pier, a foundation, a slope surface, a platform, a drainage ditch, a catch basin and the like. The inspection report content can be displayed on the data management platform in a three-dimensional model manner, for example, if inspection is performed on the bridge B, the three-dimensional model of the bridge B is built in the data management platform, and the associated position of the disease state in the inspection report on the three-dimensional model is displayed.
In some embodiments, the performing the image analysis on the video data specifically includes:
and carrying out image analysis on the video data through a pre-trained target detection network.
The training process of the target detection network comprises the following steps:
constructing training sample data, wherein the training sample data comprises a plurality of training images marked with first feature points in advance; the first feature point is an image feature of each target on the inspection object, for example, the training sample data is constructed for a bridge, and the first feature point may include image features of bridge deck, expansion joints, supports, beam bodies, piers, foundations and other objects in the underbridge space, where the other objects may include cars, buildings, garbage dumps, side slope intercepting ditches, drainage ditches, platforms, slopes, and the like. In addition, the first characteristic point also comprises position coordinates, shapes, sizes and other data of the bridge deck, the expansion joint, the support, the beam body, the pier, the foundation and other objects in the space under the bridge. The number of the training images is more than 1000.
Constructing a target detection network to be trained; the target detection network to be trained can adopt a network optimally adjusted based on a Faster-Rcnn model. The network optimized and adjusted based on the Faster-Rcnn model adopts the existing network structure.
And training the target detection network to be trained through the training sample data. During the training process, 50 epochs are set for training, the learning rate is set to 0.001, and the learning degradation rate is set to 0.1. Where epoch represents a complete training of the model using all data of the training set, also referred to as "generation training".
After the model training is completed, the video data is subjected to image processing.
In some embodiments, the image analysis of the video data by a pre-trained target detection network comprises:
carrying out target detection on a plurality of images in the video data through a pre-trained target detection network, and extracting second feature points in the plurality of images; the second feature points are physical features extracted from a plurality of images in the video data by the target detection network, such as feature data of bridge deck, expansion joints, supports, beams, piers and foundations, and other objects in the space under the bridge.
After the steps are completed, disease characteristic analysis is directly performed according to the characteristic data, and as long as the disease state existing in the bridge is analyzed, relevant characteristics, such as pit slots of the bridge deck, expansion joints, support deformation, cracks on the lower surface of the beam body, cracks on piers, foundation scouring, side slope intercepting ditches, drainage ditches, slopes and other characteristics, only need to be extracted, so that the processing process is accelerated.
And analyzing the disease state of the inspection part according to the second characteristic point.
Therefore, in the embodiment, the target detection is performed on the plurality of images in the video data through the pre-trained target detection network to obtain the plurality of feature points, so as to improve the accuracy of the feature point extraction result, and meanwhile, the relatively concentrated feature points are extracted from the plurality of feature points for analysis, so as to accelerate the disease state analysis process.
In some embodiments, the performing the image analysis on the video data may further be performed by:
carrying out image segmentation on the plurality of images to obtain a target image; the segmentation is to segment the background image and the target image in a plurality of images. In general, the gray scales of the target image and the background image on the gray scale histogram are in different gray scale intervals, so that one gray scale threshold value can be selected to segment the target image. For example, a Bradley binarization implementation in a local dynamic threshold algorithm may be employed.
Extracting the features of the target image, and extracting a third feature point of the target image; the feature extraction method of the embodiment is mainly performed by using an edge detection method. The edge exists mainly between the target and the target, between the target and the background or between the region and the region, and is a reflection of the image gray level discontinuity. The edge detection is a process representation for detecting discontinuous points of the image function, and can be realized by Canny operator in the gradient operator in the embodiment. The third feature point comprises image features with discontinuous gray levels in the detection result.
And analyzing the disease state of the highway bridge slope according to the third characteristic point.
In the embodiment, the disease state analysis of the highway bridge slope is carried out through the characteristic points obtained by image segmentation and edge detection, so that the accuracy of the analysis result is improved.
In other embodiments, after edge detection is completed, a Bag Feature model can be used for extracting features and constructing a virtual dictionary of an image, then a neural network is used for training sample data, in an image recognition stage, Feature vectors of the image are used as input of a neural network classifier, and the output of the classifier is a recognition result through network calculation.
In some embodiments, in order to improve the accuracy of image segmentation, before the step of image segmentation on the several images, the method further comprises the following steps:
respectively carrying out image enhancement on the plurality of images; in the step, the self-adaptive local enhancement processing technology is applied, only the contrast of the region of interest is enhanced, and the definition of other regions is blurred. For example, by using the laplacian operator, each gray value in the image is retained, the contrast at the abrupt change of the gray value is enhanced, and the details of the image are highlighted on the premise of retaining the background of the image.
Shadow elimination is carried out on a plurality of images after image enhancement processing; for example, shadows under sunlight irradiation on bridge poles or the like cause unevenness in image brightness, and affect automatic identification of bridge surface defects, and therefore, it is necessary to perform a process of eliminating shadows before performing defect identification. In the embodiment, the brightness elevation model can be used for modeling the brightness image, and then the hierarchical brightness compensation algorithm is used for performing brightness compensation and texture equalization on different brightness areas, so that the brightness and the details of the target image are consistent.
And filtering out noise of a plurality of images after shadow elimination. Specifically, random noise, Gaussian noise and the like of the image are filtered, and smoothing is performed before the image is divided so as to weaken the influence of the noise. This step can be implemented by a median filtering method.
In addition, referring to fig. 2, an embodiment of the present invention further provides a highway bridge slope maintenance unmanned aerial vehicle inspection system, which includes a mobile terminal operating system, an unmanned aerial vehicle, a data management platform and a server, where the mobile terminal operating system, the unmanned aerial vehicle and the data management platform are all in communication with the server, and the unmanned aerial vehicle may be further connected to the server through the mobile terminal operating system. The server is used for executing the method shown in the figure 1:
s110, acquiring inspection task data of the unmanned aerial vehicle;
s120, controlling the unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data;
s130, receiving video data uploaded by the unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection;
s140, carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part.
The contents in the method embodiments are all applicable to the system embodiments, and the beneficial effects of the method embodiments can be achieved by the system embodiments.
In some embodiments, the data management platform is used for displaying the inspection part and the disease state corresponding to the inspection part through a three-dimensional model.
Specifically, the data management platform may employ a B/S architecture schema. In the data management platform, a special database is constructed for the bridge and side slope diseases, disease data at different positions and different periods are managed in a centralized manner, association between the disease data and spatial positions is realized through a three-dimensional GIS technology, and multi-time sequence comparison analysis is carried out on the disease data at the same position.
The construction of the database is beneficial to the management of the disease life cycle, and is also convenient for discovering deep disease reasons through the accumulation of a large amount of data, thereby improving the management quality. Meanwhile, the establishment of the relational database facilitates the unified management of the attribute information such as the routing inspection object, the routing inspection time, the disease part, the disease type and the like and the photo data. The database can adopt MySQL database, but because the data traffic is large, the database can also adopt external storage and is associated by index.
The spatial position association can utilize a three-dimensional GIS technology to carry out three-dimensional display on bridges and side slopes, and spatial position information is written into a relational database in a unique coding mode by creating spatial objects so as to realize the association of disease data and spatial positions
And in a disease database, inquiring and retrieving according to the spatial position information, constructing a disease sequence in a time sequence, and performing comparative analysis through image superposition display. When a user needs to search the process of the disease state, the disease state can be sequenced according to the time sequence, the evolution process of the disease can be visually displayed, and the result can be exported for further analysis.
In some embodiments, the unmanned aerial vehicle is provided with a positioning module and a damping module, and the positioning module comprises an IMU module, a GNSS module, an optical flow sensor module, an infrared sensor module, and an RTK module. The IMU module, the GNSS module, the optical flow sensor module and the infrared sensor module can be controlled through a cascade closed loop, and the RTK module is used for realizing centimeter-level positioning.
And the innermost ring of the closed-loop control is PID attitude control based on the IMU. The IMU comprises a gyroscope and an acceleration sensor, the gyroscope is responsible for serving as an electronic compass, changes of an aircraft of the unmanned aerial vehicle in roll, yaw and pitch are measured, the acceleration sensor is responsible for measuring the acceleration of the aircraft of the unmanned aerial vehicle on the relative coordinate axis of the acceleration sensor, errors and noises are minimized through a Kalman filtering algorithm, and accurate control over an unmanned aerial vehicle platform is achieved.
On the basis of high-precision attitude control, the GNSS closed loop circuit positioned in the middle layer can exert better effect. The unmanned aerial vehicle platform uses the GNSS global satellite positioning system simultaneously, including GPS, GLONASS, big dipper, GALILEO multinational satellite positioning system, when one of them system can't normally use, other systems can continue to support task execution. On the basis of conventional satellite direct and unmanned aerial vehicle platform communication positioning, the unmanned aerial vehicle also uses the RTK system, and centimeter-level high-precision positioning is realized through the triangulation calculation among the satellite, the unmanned aerial vehicle platform and the RTK ground station.
Based on above two-layer closed loop, the unmanned aerial vehicle platform has still increased extra light stream sensor control circuit. The displacement of unmanned aerial vehicle itself is judged to the pixel displacement on the light stream sensor accessible calculation shooting plane to the stability of reverse increase aircraft. Under the condition of insufficient light, the LED light supplementing system positioned at the bottom of the aircraft platform can also increase the brightness, so that the sensor can better identify the surface texture.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. An unmanned aerial vehicle inspection method for highway bridge slope maintenance is characterized by comprising the following steps:
acquiring polling task data of the unmanned aerial vehicle;
controlling an unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data;
receiving video data uploaded by an unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection;
and carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part.
2. The unmanned aerial vehicle inspection method for highway bridge slope maintenance according to claim 1, wherein the image analysis is performed on the video data, and specifically comprises:
performing image analysis on the video data through a pre-trained target detection network;
the training process of the target detection network comprises the following steps:
constructing training sample data, wherein the training sample data comprises a plurality of training images marked with first feature points in advance;
constructing a target detection network to be trained;
and training the target detection network to be trained through the training sample data.
3. The method for unmanned aerial vehicle inspection according to claim 2, wherein the image analysis of the video data through a pre-trained target detection network comprises:
carrying out target detection on a plurality of images in the video data through a pre-trained target detection network, and extracting second feature points in the plurality of images;
and analyzing the disease state of the inspection part according to the second characteristic point.
4. The method for unmanned aerial vehicle inspection according to claim 1, wherein the performing image analysis on the video data comprises:
carrying out image segmentation on the plurality of images to obtain a target image;
extracting the features of the target image, and extracting a third feature point of the target image;
and analyzing the disease state of the highway bridge slope according to the third characteristic point.
5. The unmanned road inspection method for highway bridge slope maintenance according to claim 4, wherein before the step of image segmentation of the images, the method further comprises the following steps of:
respectively carrying out image enhancement on the plurality of images;
shadow elimination is carried out on a plurality of images after image enhancement processing;
and filtering out noise of a plurality of images after shadow elimination.
6. The method for unmanned aerial vehicle inspection according to claim 1, wherein the step of controlling the unmanned aerial vehicle to perform the inspection of the highway bridge slope according to the inspection task data comprises the steps of:
controlling the flight path of the unmanned aerial vehicle according to the routing inspection path data in the routing inspection task data;
acquiring real-time position data of the unmanned aerial vehicle during flight;
calculating the linear distance between the real-time position data and target position data in the routing inspection task data;
after the linear distance is determined to belong to the preset range, the flight speed of the unmanned aerial vehicle and the shooting angle and the shooting mode of a camera on the unmanned aerial vehicle are adjusted, and video data are acquired for the target position corresponding to the current target position data.
7. The utility model provides a highway bridge side slope maintenance unmanned aerial vehicle system of patrolling and examining, its characterized in that, including removing end operating system, unmanned aerial vehicle, data management platform and server, remove end operating system, unmanned aerial vehicle and data management platform all with the server communication, the server is used for carrying out following step:
acquiring polling task data of the unmanned aerial vehicle;
controlling an unmanned aerial vehicle to carry out highway bridge slope inspection according to the inspection task data;
receiving video data uploaded by an unmanned aerial vehicle, wherein the video data are a plurality of images acquired when the unmanned aerial vehicle executes the highway bridge slope inspection;
and carrying out image analysis on the video data to generate an inspection report, wherein the inspection report comprises an inspection part and a disease state corresponding to the inspection part.
8. The inspection system according to claim 7, wherein the data management platform is configured to display the inspection part and the disease state corresponding to the inspection part through a three-dimensional model.
9. The inspection system according to claim 8, wherein the data management platform is further configured to build a disease state library of the highway bridge slopes and perform time-sequence management on the disease states.
10. According to go there and require 7 a highway bridge side slope maintenance unmanned aerial vehicle system of patrolling and examining, its characterized in that is equipped with orientation module and shock-absorbing module on the unmanned aerial vehicle, orientation module includes IMU module, GNSS module, light stream sensor module, infrared sensor module and RTK module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011000233.3A CN112215805A (en) | 2020-09-22 | 2020-09-22 | Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011000233.3A CN112215805A (en) | 2020-09-22 | 2020-09-22 | Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112215805A true CN112215805A (en) | 2021-01-12 |
Family
ID=74049744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011000233.3A Pending CN112215805A (en) | 2020-09-22 | 2020-09-22 | Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215805A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113160022A (en) * | 2021-04-27 | 2021-07-23 | 广东天濠建设工程有限公司 | Municipal bridge maintenance management system, method and equipment and readable storage medium |
| CN113359832A (en) * | 2021-06-17 | 2021-09-07 | 上海大风技术有限公司 | Bridge detection system and detection method based on unmanned aerial vehicle operation |
| CN113673722A (en) * | 2021-08-27 | 2021-11-19 | 招商局重庆交通科研设计院有限公司 | Accurate decision making system and method for maintenance of expansion joint |
| CN114167889A (en) * | 2021-11-29 | 2022-03-11 | 内蒙古易飞航空科技有限公司 | Intelligent inspection flight platform based on image AI and big data application |
| CN114281107A (en) * | 2021-12-21 | 2022-04-05 | 河南高建工程管理有限公司 | Highway slope maintenance inspection method |
| CN114612473A (en) * | 2022-05-11 | 2022-06-10 | 深圳联和智慧科技有限公司 | Bridge inspection processing method and system based on unmanned aerial vehicle and cloud platform |
| CN114998772A (en) * | 2022-08-03 | 2022-09-02 | 深圳联和智慧科技有限公司 | Integrated bridge detection method and system based on unmanned aerial vehicle and cloud platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130216089A1 (en) * | 2010-04-22 | 2013-08-22 | The University Of North Carolina At Charlotte | Method and System for Remotely Inspecting Bridges and Other Structures |
| CN109885098A (en) * | 2019-04-11 | 2019-06-14 | 株洲时代电子技术有限公司 | A kind of bridge sidebar inspection flight course planning method |
| CN111260615A (en) * | 2020-01-13 | 2020-06-09 | 重庆交通大学 | Laser and machine vision fusion-based method for detecting apparent diseases of unmanned aerial vehicle bridge |
| CN111340772A (en) * | 2020-02-24 | 2020-06-26 | 湖北工业大学 | Reinforced concrete bridge damage detection system and method based on mobile terminal |
| CN111524121A (en) * | 2020-04-22 | 2020-08-11 | 南京智行信息科技有限公司 | Road and bridge fault automatic detection method based on machine vision technology |
-
2020
- 2020-09-22 CN CN202011000233.3A patent/CN112215805A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130216089A1 (en) * | 2010-04-22 | 2013-08-22 | The University Of North Carolina At Charlotte | Method and System for Remotely Inspecting Bridges and Other Structures |
| CN109885098A (en) * | 2019-04-11 | 2019-06-14 | 株洲时代电子技术有限公司 | A kind of bridge sidebar inspection flight course planning method |
| CN111260615A (en) * | 2020-01-13 | 2020-06-09 | 重庆交通大学 | Laser and machine vision fusion-based method for detecting apparent diseases of unmanned aerial vehicle bridge |
| CN111340772A (en) * | 2020-02-24 | 2020-06-26 | 湖北工业大学 | Reinforced concrete bridge damage detection system and method based on mobile terminal |
| CN111524121A (en) * | 2020-04-22 | 2020-08-11 | 南京智行信息科技有限公司 | Road and bridge fault automatic detection method based on machine vision technology |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113160022A (en) * | 2021-04-27 | 2021-07-23 | 广东天濠建设工程有限公司 | Municipal bridge maintenance management system, method and equipment and readable storage medium |
| CN113359832A (en) * | 2021-06-17 | 2021-09-07 | 上海大风技术有限公司 | Bridge detection system and detection method based on unmanned aerial vehicle operation |
| CN113673722A (en) * | 2021-08-27 | 2021-11-19 | 招商局重庆交通科研设计院有限公司 | Accurate decision making system and method for maintenance of expansion joint |
| CN113673722B (en) * | 2021-08-27 | 2023-11-24 | 招商局重庆交通科研设计院有限公司 | Precise decision system and method for expansion joint maintenance opportunity |
| CN114167889A (en) * | 2021-11-29 | 2022-03-11 | 内蒙古易飞航空科技有限公司 | Intelligent inspection flight platform based on image AI and big data application |
| CN114167889B (en) * | 2021-11-29 | 2023-03-07 | 内蒙古易飞航空科技有限公司 | Intelligent inspection flight platform based on image AI and big data application |
| CN114281107A (en) * | 2021-12-21 | 2022-04-05 | 河南高建工程管理有限公司 | Highway slope maintenance inspection method |
| CN114612473A (en) * | 2022-05-11 | 2022-06-10 | 深圳联和智慧科技有限公司 | Bridge inspection processing method and system based on unmanned aerial vehicle and cloud platform |
| CN114612473B (en) * | 2022-05-11 | 2022-08-16 | 深圳联和智慧科技有限公司 | Bridge inspection processing method and system based on unmanned aerial vehicle and cloud platform |
| CN114998772A (en) * | 2022-08-03 | 2022-09-02 | 深圳联和智慧科技有限公司 | Integrated bridge detection method and system based on unmanned aerial vehicle and cloud platform |
| CN114998772B (en) * | 2022-08-03 | 2022-11-18 | 深圳联和智慧科技有限公司 | Integrated bridge detection method and system based on unmanned aerial vehicle and cloud platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112215805A (en) | Unmanned aerial vehicle inspection method and system for highway bridge slope maintenance | |
| Polvara et al. | Toward end-to-end control for UAV autonomous landing via deep reinforcement learning | |
| EP3759562B1 (en) | Camera based localization for autonomous vehicles | |
| Li et al. | Automatic bridge crack detection using Unmanned aerial vehicle and Faster R-CNN | |
| CN103389103B (en) | A kind of Characters of Geographical Environment map structuring based on data mining and air navigation aid | |
| US20220383484A1 (en) | Tunnel defect detecting method and system using unmanned aerial vehicle | |
| KR20210111180A (en) | Method, apparatus, computing device and computer-readable storage medium for positioning | |
| CN110196454B (en) | Geological survey integrated system based on unmanned aerial vehicle | |
| CN112734765B (en) | Mobile robot positioning method, system and medium based on fusion of instance segmentation and multiple sensors | |
| CA3002308A1 (en) | Device and method for autonomous localisation | |
| CN109885097B (en) | Method for planning inspection route of outer edge surface of bridge | |
| US20210112376A1 (en) | Sensitive Area Management | |
| CN109885098B (en) | Method for planning inspection route of bridge side fence | |
| CN112308913B (en) | Vehicle positioning method and device based on vision and vehicle-mounted terminal | |
| CN112102403B (en) | High-precision positioning method and system for autonomous inspection unmanned aerial vehicle in power transmission tower scene | |
| Xing et al. | Multi-UAV cooperative system for search and rescue based on YOLOv5 | |
| US20230286556A1 (en) | Autonomous drone for railroad track inspection | |
| CN109764869A (en) | A kind of positioning of autonomous crusing robot and the three-dimensional map construction method of binocular camera and inertial navigation fusion | |
| CN116647651B (en) | Unmanned aerial vehicle construction monitoring method and system based on Beidou satellite | |
| CN115639823A (en) | Terrain sensing and movement control method and system for robot under rugged and undulating terrain | |
| CN113238251A (en) | Target-level semantic positioning method based on vehicle-mounted laser radar | |
| CN113674355A (en) | Target identification and positioning method based on camera and laser radar | |
| CN116605772A (en) | Tower crane collision early warning method based on multiple integrated systems | |
| CN115496399B (en) | Unmanned aerial vehicle-based foundation pit survey task instant updating and distributing method and system | |
| CN110968112A (en) | Unmanned aerial vehicle autonomous landing system and method based on monocular vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |