CN112098326B - Automatic detection method and system for bridge diseases - Google Patents
Automatic detection method and system for bridge diseases Download PDFInfo
- Publication number
- CN112098326B CN112098326B CN202010842289.7A CN202010842289A CN112098326B CN 112098326 B CN112098326 B CN 112098326B CN 202010842289 A CN202010842289 A CN 202010842289A CN 112098326 B CN112098326 B CN 112098326B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- detection
- bridge
- disease
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
- G01N2021/0106—General arrangement of respective parts
- G01N2021/0112—Apparatus in one mechanical, optical or electronic block
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Pathology (AREA)
- Immunology (AREA)
- Remote Sensing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Automation & Control Theory (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an automatic detection method and system aiming at bridge diseases, wherein the bridge detection process is divided into three detection levels of 'whole-member-local', the detection work is implemented through a multi-motion mode intelligent unmanned aerial vehicle system which has air flight and is adsorbed at the hidden part of a bridge bottom/bridge tower, an unmanned aerial vehicle is autonomously controlled by adopting an adaptive inspection route dynamic programming algorithm based on information feedback to obtain bridge surface images and sound wave feedback information under different scales, then bridge disease information is automatically excavated and analyzed through a multi-scale deep learning network corresponding to the three-level detection scales, and the accurate detection of internal and external diseases of the hidden part of a bridge including the bridge bottom and the like is realized from optimal measurement and optimal analysis. Compared with the traditional manual detection, the method provided by the invention has the advantages that the cost is greatly reduced, the detection efficiency is obviously improved, the multi-level detection method can comprehensively and finely cover the surface and internal information of the structure, the detection information is comprehensive and complete, and the traffic is not influenced.
Description
Technical Field
The invention relates to the technical field of intelligent signal processing, in particular to an automatic detection method and system for bridge diseases.
Background
The total number of bridges in China exceeds 100 ten thousand, and the bridge is in safe service relation with national civilian life. The structure detection is an important means for guaranteeing the safety of the bridge, and the clear requirement of the compendium of traffic strong nations ' is ' strengthening the monitoring and detection of infrastructure operation and greatly promoting the deep fusion of new technologies such as big data, Internet of things, artificial intelligence and the like with the traffic industry '.
The traditional bridge bottom disease detection method depends on manual visual detection and large-scale bridge detection equipment, and has the problems of high cost, low efficiency, traffic obstruction and the like. In order to overcome the problem of manual detection, computer vision and deep learning technologies are applied to structural disease detection in recent years, and a good application prospect is shown. However, because the depth of a common convolutional network model is deep and the parameter quantity is large, most of current disease detection methods based on deep learning are offline detection, disease pictures or videos need to be collected in field industry first, and then diseases are detected through field industry analysis, and the detection mode cannot meet the high-efficiency and quick requirements in engineering detection. Meanwhile, most of the existing detection methods based on intelligent detection platforms such as unmanned aerial vehicles detect surface diseases, and damage inside the structure cannot be identified.
On the other hand, the unmanned aerial vehicle system is also beginning to be applied to structural detection more and more as a cheap and convenient working platform. However, the existing unmanned aerial vehicle has great limitation in being applied to bridge bottom defect detection. Firstly, the traditional unmanned aerial vehicles all rely on GPS information to realize unmanned aerial vehicle positioning, and the unmanned aerial vehicles at the bottom of the bridge cannot normally work because the unmanned aerial vehicles cannot receive the GPS information due to being shielded; in addition, traditional unmanned aerial vehicle need pay attention to constantly in order to avoid colliding with bridge bottom, pier etc. when the bridge bottom work, and this has proposed strict requirement to operating personnel. Because traditional unmanned aerial vehicle must keep certain safe distance with the structure when detecting the crack in order to prevent colliding, consequently be difficult to acquire the careful crack photo in structure surface, still there is unmanned aerial vehicle to guarantee when shooing that camera and surveyed the face parallel, cause the disease photo that acquires to have the slope deformation scheduling problem that is difficult to eliminate. The practical engineering application of the unmanned aerial vehicle platform and the deep learning method in the structural disease detection is greatly limited by the problems.
Disclosure of Invention
Aiming at the problems, the invention provides an automatic detection method and system for bridge diseases.
The method comprises the steps of dividing bridge detection into three layers of whole bridge, dispersed components and local diseases, using a multi-mode unmanned aerial vehicle with flight and adsorption crawling as a platform for implementing detection, guiding the unmanned aerial vehicle to automatically expand inspection by a dynamic inspection path automatic planning method based on real-time detection result feedback in the detection process of each layer, and analyzing and mining the disease characteristics in the obtained information through a multi-layer deep learning network corresponding to the scale of the three-layer detection characteristics after inspection is finished.
In one embodiment, the disease automatic analysis mining method of the multi-level deep learning network divides the whole network into an independent feature extraction layer (parallel connection) and a shared feature prediction layer (series connection) through series-parallel connection; the system comprises a feature extraction layer, a plurality of sets of convolution layers, a batch normalization layer, an activation layer and a residual layer, wherein the feature extraction layer respectively designs three parallel feature extraction sub-networks according to the whole scale, the component scale and the local disease scale of a bridge, the structure of a single sub-network is similar to that of a common feature extraction network and comprises the plurality of sets of convolution layers, the batch normalization layer, the activation layer and the residual layer, for large-scale input, the residual layers are more, the deeper the network is, and the larger the down-sampling multiplying power is; for the feature prediction layer, the idea similar to a single-shot multi-frame detector is adopted, a certain number of candidate frames with different length-width ratios are obtained through clustering analysis, then dense sampling is uniformly carried out at different positions of a feature map, and classification and regression are directly carried out.
In one embodiment, the self-adaptive routing inspection dynamic route planning method based on information feedback is divided into two parts, namely whole-component stage planning and component-local stage planning according to detection levels, for whole level detection, the core is to abstract the trend of a bridge component according to the geometric shape of a bridge, judge the video shot by an unmanned aerial vehicle carrying camera by using a classification neural network so as to identify the pointing direction and the spatial position of the unmanned aerial vehicle relative to the bridge component at each moment, further guide the unmanned aerial vehicle to automatically inspect according to the component trend, and simultaneously identify each component of the bridge and record the spatial position, and use the spatial position as an interest point; for component and local level detection, the core of the method is to convert the optimization problem of the optimal route for unmanned aerial vehicle routing inspection into a mathematical TSP problem according to the interest points detected in the previous level, namely, firstly, a state transfer function go (S, init) is established according to the time loss and the Manhattan distance of the interest points, wherein the min { go (S-i, i) + dp [ i ] [ init ] }, and then a Hamilton loop with the minimum weight is searched through a fast search random tree algorithm, so that the total distance go (S, init) traversing all the interest points S from the first init point is minimum, and the unmanned aerial vehicle routing inspection route is shortest and the efficiency is highest.
In one embodiment, the intelligent drone system includes a flight module, an adsorption module, and a detection module;
the flight module comprises a lift system in an X-type layout with the equal space of 550mm and a navigation control system based on multi-sensor fusion, and is used for realizing six-degree-of-freedom flight of the unmanned aerial vehicle; the adsorption module comprises a transverse adsorption motor and a forward-inclined frame which is specially designed and is used for enabling the unmanned aerial vehicle to be adsorbed on the vertical face or the bottom face of the bridge to move and acquiring close-range detailed disease information; the detection module is used for detecting bridge disease information.
In one embodiment, the intelligent drone system includes a positioning system; the positioning system comprises a GPS positioning system and a visual navigation system, the GPS positioning system provides positioning information when GPS signals exist, and the visual navigation system provides the positioning information automatically when no GPS signals exist.
In an embodiment, the automatic detection method for bridge diseases further includes:
when the unmanned aerial vehicle adsorbs the structure surface of disease position department, the contact scanning that unmanned aerial vehicle carried on strikes echo system and strikes the test in succession to disease position, obtains crack depth, inside damage and concrete degradation.
The internal damage detection method comprises a vibration exciter for continuously knocking the surface of the structure and a receiver which is attached to the surface of the structure and rolls to collect impact echo signals, and the unmanned aerial vehicle moves and knocks while continuously collecting the echo signals in the moving process of the surface of the adsorption structure.
In one embodiment, the automatic detection method for bridge diseases includes the following steps:
s10, carrying out overall hierarchical coarse detection, automatically planning a route by an intelligent unmanned aerial vehicle system according to the trend of the bridge, identifying components of each part of the bridge by continuously shooting a large number of images of each part of the bridge with a multi-angle and large view field, marking the position of each component according to the positioning information of the unmanned aerial vehicle, and taking the position as a key interest point for second-level route planning;
and S20, performing hierarchical detection on the components, planning a detection route by taking the position of the component detected in the previous step as a key interest point and taking the shortest route as a principle, flying the unmanned aerial vehicle according to the detection route, acquiring image information of a small view field on the surface of the component through a carried camera when the unmanned aerial vehicle flies to one detection route, and detecting diseases according to the acquired images. When a disease is detected, recording the position of the detected disease through the positioning information of the unmanned aerial vehicle;
s30, local level precise detection, planning a third level routing inspection route by taking the disease position detected in the previous step as an interest point, and during the third level fine disease detection, adsorbing the structure surface at the disease position by an unmanned aerial vehicle, collecting a disease image and the internal damage information of the disease surface, and directly obtaining the geometric information of the disease through the analysis of a recurrent neural network.
The automatic detection method and the system aiming at the bridge diseases can continuously shoot images of each part of the bridge at multiple angles, identify components of each part of the bridge according to the images of each part of the bridge, mark the positions of the components of each part according to the positioning information of the unmanned aerial vehicle, take the positions of the components of each part as key interest points of second-level route planning, plan a detection route according to the key interest points and the shortest route as the principle, the unmanned aerial vehicle flies according to the detection route, when flying to a detection route, acquire the large-field image information of the surfaces of the components through the carried camera, detect the diseases according to the image information of the surfaces of the components, when the diseases are detected, record the positions of the detected diseases through the positioning information of the unmanned aerial vehicle, plan a routing inspection route of a third level by taking the positions of the diseases as the interest points, and when the diseases are detected in a third-level, the unmanned aerial vehicle is adsorbed to the surface of the structures at the positions of the diseases, gather disease image and disease surface depth information to directly obtain the geometric information of disease through the analysis of multiscale neural network, with the automated inspection who realizes the bridge disease, wherein do not need to rely on artifical control and manual analysis, can independently patrol and examine, independently discern the disease, by whole to the component to detect in layers partially, maximize detection efficiency and detection integrality, compare with traditional manual detection and promote detection efficiency greatly when reduce cost by a wide margin, and have and do not influence advantages such as traffic.
Drawings
FIG. 1 is a flow chart of an embodiment of a method for automatic detection of bridge diseases;
fig. 2 is a schematic structural diagram of an intelligent drone system of one embodiment;
fig. 3 is a framework diagram of an intelligent drone system of one embodiment;
fig. 4 is a schematic diagram of an embodiment of an intelligent drone system operation;
fig. 5 is a schematic diagram of a three-hierarchy disease identification network based on multi-scale deep learning according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, fig. 1 is a flowchart of an automatic detection method for bridge diseases according to an embodiment, and includes the following steps:
s10, carrying out overall level coarse detection, automatically planning a route by the intelligent unmanned aerial vehicle system according to the trend of the bridge, identifying components of each part of the bridge by continuously shooting a large number of images of each part of the bridge with a large multi-angle and large view field, marking the position of each component according to the positioning information of the unmanned aerial vehicle, and taking the position as a key interest point for second level route planning.
The steps are that firstly, the detection of the whole layer of the bridge is carried out, and the unmanned aerial vehicle automatically plans a route according to the trend of the bridge. A large number of multi-angle images of all parts of the bridge are continuously shot, all parts of the bridge are identified, the positions of all parts are marked according to the positioning information of the unmanned aerial vehicle, and the positions are used as key interest points of second-level route planning.
In one embodiment, the continuously capturing images of each part of the bridge from multiple angles, identifying each part of the bridge according to the images of each part of the bridge, and marking the position of each part of the bridge according to the positioning information of the unmanned aerial vehicle, wherein the step of taking the position of each part of the bridge as a key interest point of the second-level route planning includes:
abstracting the trend of the bridge part components according to the geometric shapes of the bridge, judging the pointing direction and the spatial position of the unmanned aerial vehicle relative to the part components at each moment by utilizing the classification neural network to shoot videos of the unmanned aerial vehicle carrying camera, further guiding the unmanned aerial vehicle to automatically patrol according to the trend of the part components, identifying each part component of the bridge and recording the position of each part component, and taking the position of each part component as a key interest point of second-level route planning.
The method is characterized in that the trend of a bridge member is abstracted according to the geometric shape of the bridge, a classification neural network is used for judging a video shot by a camera carried by the unmanned aerial vehicle so as to identify the pointing direction and the spatial position of the unmanned aerial vehicle relative to the bridge member at each moment, further the unmanned aerial vehicle is guided to automatically inspect according to the trend of the member, meanwhile, each member of the bridge is identified, the spatial position is recorded, and the spatial position is used as an interest point; for component and local level detection, the core of the method is to convert the optimization problem of the optimal route for unmanned aerial vehicle routing inspection into a mathematical TSP problem according to the interest points detected in the previous level, namely, firstly, a state transfer function go (S, init) is established according to the time loss and the Manhattan distance of the interest points, wherein the min { go (S-i, i) + dp [ i ] [ init ] }, and then a Hamilton loop with the minimum weight is searched through a fast search random tree algorithm, so that the total distance go (S, init) traversing all the interest points S from the first init point is minimum, and the unmanned aerial vehicle routing inspection route is shortest and the efficiency is highest.
Specifically, the multi-scale parallel multi-branch deep learning network designed in this embodiment is a series-parallel combined network structure, and the whole network is divided into two parts, namely an independent feature extraction layer (parallel connection) and a shared feature prediction layer (series connection), by series-parallel connection. For the feature extraction layer, because the input feature scales are greatly different, three parallel feature extraction sub-networks are respectively designed according to the overall scale of the bridge, the component scale and the local disease scale. The structure of a single sub-network is similar to that of a common feature extraction network, and the single sub-network consists of a plurality of convolution layers, a batch normalization layer, an activation layer and a residual error layer. In contrast, for a large-scale input, the residual layers are more, the deeper the network is, and thus the larger the down-sampling magnification is. By setting a proper number of residual error networks and up-sampling steps, the sizes of final feature maps extracted from input of different scales are approximately the same, and therefore the feature maps with similar scales can be output through shared feature prediction layer processing, namely after multi-scale feature images obtained by three-level detection are input. For the feature prediction layer, the idea similar to a single-shot multi-frame detector is adopted, a certain number of candidate frames with different length-width ratios are obtained through clustering analysis, then dense sampling is uniformly carried out at different positions of a feature map, and classification and regression are directly carried out. The problem of inconsistent detection of targets with different scales can be solved by combining the multi-scale feature extraction layer connected in parallel and the feature prediction layer connected in series, and the accurate detection of the whole-member-local multi-level diseases of the structure is realized.
Furthermore, the designed multi-scale parallel multi-branch deep learning network is a lightweight network, and the parameters and complexity of the model are reduced by adopting a depth reelable and chained architecture in a large amount, so that the model can directly run in an onboard computer carried by the unmanned aerial vehicle, the structural surface image shot by the unmanned aerial vehicle camera is processed in real time, and online detection is realized.
And S20, performing hierarchical detection on the components, planning a detection route by taking the position of the component detected in the previous step as a key interest point and taking the shortest route as a principle, flying the unmanned aerial vehicle according to the detection route, acquiring image information of a small view field on the surface of the component through a carried camera when the unmanned aerial vehicle flies to one detection route, and detecting diseases according to the acquired images. When detecting the disease, the disease position that detects is recorded through unmanned aerial vehicle's locating information.
The steps can automatically plan a detection route according to the key interest points and the shortest route, so that the unmanned aerial vehicle automatically flies according to the route. When the unmanned aerial vehicle flies to a component position, images and depth information of the surface of the component are collected through the carried camera, and diseases are identified in real time. When discerning the disease, through unmanned aerial vehicle's locating information record disease position.
S30, local level precise detection, planning a third level routing inspection route by taking the disease position detected in the previous step as an interest point, and during the third level fine disease detection, adsorbing the structure surface at the disease position by an unmanned aerial vehicle, collecting a disease image and the internal damage information of the disease surface, and directly obtaining the geometric information of the disease through the analysis of a recurrent neural network.
And in the step, aiming at the disease position positioned above, the disease position is used as an interest point to plan a route of the third-level inspection. When the third-level fine disease detection is carried out, the unmanned aerial vehicle adsorbs to the surface of the structure, acquires fine disease images and disease surface depth information, and directly obtains the geometric information of the diseases through high-dimensional neural network analysis. Meanwhile, a contact type scanning impact echo system carried by the unmanned aerial vehicle carries out continuous knocking test on the disease position to obtain disease information such as crack depth, internal damage and concrete deterioration.
The automatic detection method for the bridge diseases can continuously shoot images of each part of the bridge at multiple angles, identify components of each part of the bridge according to the images of each part of the bridge, mark the positions of the components of each part according to the positioning information of the unmanned aerial vehicle, take the positions of the components of each part as key interest points for second-level route planning, plan a detection route according to the key interest points and the shortest route as the principle, the unmanned aerial vehicle flies according to the detection route, acquire images and depth information of the surfaces of the components through a camera carried on the unmanned aerial vehicle when flying to one detection route, detect the diseases according to the images and the depth information of the surfaces of the components, record the positions of the detected diseases through the positioning information of the unmanned aerial vehicle when the diseases are detected, plan a routing inspection route of a third level by taking the positions of the diseases as the interest points, and the unmanned aerial vehicle adsorbs to the surface of the structures at the positions of the diseases when the diseases are detected in a fine disease detection of the third level, gather disease image and disease surface depth information to directly obtain the geometric information of disease through high dimension neural network analysis, with the automated inspection who realizes the bridge disease, wherein do not need to rely on artifical control and manual analysis, can independently patrol and examine, independently discern the disease, by whole to the component to detect in layers partially, maximize detection efficiency and detection integrality, compare with traditional manual detection and promote detection efficiency greatly when reduce cost by a wide margin, and have and do not influence advantages such as traffic.
In an embodiment, the automatic detection method for bridge diseases further includes:
when the unmanned aerial vehicle adsorbs the structure surface of disease position department, the contact scanning that unmanned aerial vehicle carried strikes echo system and strikes the test in succession to the disease position, obtains crack depth, internal damage and concrete degradation.
Specifically, the internal damage detection method comprises a vibration exciter for continuously knocking the surface of the structure and a receiver which is attached to the surface of the structure and rolls to collect impact echo signals, and the unmanned aerial vehicle can continuously collect the echo signals while moving and knocking in the process of moving the surface of the adsorption structure.
In this embodiment, the internal damage detection method is a detection method based on separation of variable frequency pulse excitation and harmonic wave characteristics. The vibration exciter for continuously knocking the surface of the structure and the receiver for collecting impact echo signals by rolling on the surface of the structure are attached, and the unmanned aerial vehicle can continuously collect the echo signals while moving and knocking in the process of moving on the surface of the adsorption structure. For damages such as concrete cracks and internal holes, reflected waves are generated when the exciting waves reach the damaged surface, and the reflected waves are received by a receiver and then converted into a frequency domain through fast Fourier transform, so that the frequency peak value of the reflected waves is determined. The defect depth can be calculated as D ═ β × V p ) Where β is the shape factor, V is determined by the component type p Is the propagation speed of sound wave in the material; f is the calculated dominant frequency.
In practical application, the automatic detection method for the bridge diseases can give consideration to both high efficiency and completeness of detection, and gradually expand the whole bridge structure detection process from three layers of 'whole-member-local' in stages. Because the detection range of the whole level and the component level is wide, the detection speed is high, and the detection is relatively rough, the unmanned aerial vehicle carries out scanning detection in a common flight state, and automatic identification of diseases is carried out by utilizing a multi-scale feature fusion deep learning network designed aiming at the scale change of an object in the detection process. For local level detection, the unmanned aerial vehicle adopts internal damage sound wave excitation type detection of adsorption crawling state expansion.
The corresponding drone comprises three flight states: a normal flight state, a facade adsorption state and a top adsorption state. In a common flight state, the adsorption motor does not work, the lift force motor provides lift force, and the flight mode of the unmanned aerial vehicle is the same as that of a common quad-rotor unmanned aerial vehicle; when facade absorption state, the installation direction through making anterior horizontal absorption motor and flexible supporting wheel be theta with vertical direction and be 5 contained angles, make unmanned aerial vehicle be the state of slightly inclining forward at the fuselage in the twinkling of an eye that contacts the structure facade, therefore in the twinkling of an eye that unmanned aerial vehicle contacted the structure surface, vertical lift motor can produce horizontal component F simultaneously Vx ＝F V sin theta, push away unmanned aerial vehicle tightly and adsorb in structure facade, avoid adsorbing unmanned aerial vehicle platform unstability in the twinkling of an eye. At the moment, the up-down movement of the unmanned aerial vehicle depends on the vertical component force F of the lift motor Vy ＝F V The cos theta is controlled, when the component force is larger than the gravity G, the unmanned aerial vehicle climbs, and otherwise, the unmanned aerial vehicle descends; during the top adsorption state, lift motor provides lift and adsorption affinity, provides the thrust of equal size or differential size by adsorbing the motor and promotes unmanned aerial vehicle and remove.
Compared with the related traditional scheme, the automatic detection method for the bridge diseases has the advantages that:
(1) the provided bridge disease automatic detection method achieves 'pertinence' on effective detection of the key structural diseases from two angles of optimal measurement and optimal analysis, and greatly improves detection efficiency. In the aspect of a measuring method, the method comprises an intelligent unmanned aerial vehicle platform and an unmanned aerial vehicle routing inspection route self-adaptive dynamic planning method based on real-time feedback of a detection result, and realizes overall level coarse detection, component level fine detection and local level fine detection of a bridge structure; in the aspect of an analysis method, a three-level structure disease identification network corresponding to bridge three-level detection and based on multi-scale deep learning automatically identifies multi-level diseases, and an automatic sound wave analysis method based on a circulating neural network accurately and automatically identifies internal damages (such as internal cavities, steel bar corrosion and the like) of a structure during local detailed detection.
(2) The intelligent unmanned aerial vehicle system operating the automatic detection method for the bridge diseases is completely different from the existing unmanned aerial vehicle platform, and has two working states of common flight and adsorption type crawling, the two states are combined for detection, so that the detection efficiency is guaranteed, a high-precision detection platform can be provided when detailed detection is needed, and more detailed surface disease images can be acquired compared with the traditional unmanned aerial vehicle. In addition, contact adsorbs to detect provides feasible platform for impacting echo detection, makes unmanned aerial vehicle can detect the surface disease, can detect inside damage again.
(3) The control mode of absorption formula unmanned aerial vehicle system wherein is totally different with current unmanned aerial vehicle, and it adopts self-adaptation, independently patrols and examines the mode, need not artificial intervention completely, provides the location by the visual navigation system in the unmanned aerial vehicle system on the one hand, and multisensor information fusion provides keeps away the barrier. And on the other hand, the automatic route planning of the unmanned aerial vehicle adopts a self-adaptive automatic planning method, the inspection process is divided into three layers from the whole to the component to the local diseases, the detection result of each layer provides key interest points for the detection of the next layer, and the interest points are connected in series according to the principle of the shortest path to carry out route planning. This kind of control the mode and need not manual intervention, has avoided traditional unmanned aerial vehicle to need skilled control personnel's restriction, and automatic efficiency of patrolling and examining is higher than manual operation efficiency.
(4) Wherein absorption formula unmanned aerial vehicle system does not have GPS signal under to detection environment such as bridge bottom, ordinary unmanned aerial vehicle can't normally work, bridge bottom environment is complicated simultaneously for traditional unmanned aerial vehicle controls the difficulty, dangerous high problem, an unmanned aerial vehicle system based on vision guide is proposed, through adopting two mesh vision/be used to lead the unmanned aerial vehicle location under the fusion algorithm realization bridge bottom does not have GPS environment such as, and realize unmanned aerial vehicle with multiple sensor information fusion such as ultrasonic ranging and keep away the barrier.
(5) The disease detection method provided by the invention aims at the scale change characteristic that a detection object is changed from a large-size member (>5m) to a large-scale disease (1 m-5 m) and then to a local disease (1m) in the whole to local three-level detection process, and realizes the simultaneous extraction and prediction of the characteristics of different-scale objects by designing a corresponding three-level structure disease identification network based on multi-scale deep learning.
(6) The internal damage detection system realizes internal damage detection on the unmanned aerial vehicle platform, and the scanning type impact echo device can obtain information such as a disease distribution diagram and disease depth at a specific position of the structure. The idea of integrated external disease and internal damage comprehensive detection can obtain more comprehensive and more critical parameters for structural performance evaluation, and can provide more powerful data support for the safety evaluation of the whole structure
In one embodiment, an automatic detection system for bridge diseases is provided, which comprises an intelligent unmanned aerial vehicle system with two detection modes, namely a flight mode and an adsorption crawling mode; the intelligent unmanned aerial vehicle system executes the automatic detection method for the bridge diseases in any one of the embodiments.
In one embodiment, the intelligent drone system includes a flight module, an adsorption module, and a detection module;
the flight module comprises four vertical lift motors, and the four vertical lift motors adopt an equal-wheelbase four-rotor layout and are used for realizing six-degree-of-freedom flight of the unmanned aerial vehicle and acquiring a long-distance large-field-of-view image on the surface of the bridge deck; the adsorption module comprises two adsorption motors and a forward-inclined frame which is specially designed and is used for enabling the unmanned aerial vehicle to be adsorbed on the vertical face or the bottom face of the bridge to move so as to obtain close-range detailed disease information; the detection module is used for detecting bridge defect information.
The automatic detection system for bridge diseases provided by the embodiment comprises an intelligent unmanned aerial vehicle system with two detection modes of a flight mode and an adsorption crawling mode, an adaptive inspection dynamic route planning method based on information feedback and an automatic bridge disease mining processing method based on a multi-scale deep learning network; the intelligent unmanned aerial vehicle system comprises a flight module, an adsorption module and a detection module, wherein the flight module is provided with four vertical lift motors, and adopts an equiaxed-distance four-rotor layout, so that six-degree-of-freedom flight of the unmanned aerial vehicle can be realized, and long-distance large-view-field images on the surface of a bridge deck can be acquired. The adsorption module comprises two adsorption motors and a forward-inclined frame which is specially designed, so that the unmanned aerial vehicle can be adsorbed on the vertical face or the bottom face of the bridge to move, and close-range detailed disease information can be acquired. The self-adaptive routing inspection dynamic route planning method based on information feedback can realize the overall rough detection, the component fine detection and the local fine detection of the bridge structure and position navigation under the condition of no GPS (global positioning system) at the bottom of the bridge and the like. The automatic bridge disease excavation processing method based on the multi-scale deep learning network comprises surface disease detection and internal damage detection, wherein the core of the surface disease detection is that a series-parallel deep learning network with scale change coping capability is used for automatically analyzing an integral-member-local multi-scale detection image, and the core of the internal damage detection is that knocking detection is carried out on a possibly damaged part by using a method based on variable frequency pulse excitation and sound wave characteristic separation. The whole set of system can realize autonomous and self-adaptive detection of the surface diseases and the internal damages of the bridge, does not need manual intervention, has high detection efficiency and wide engineering application prospect. In one example, a schematic structural diagram of the intelligent drone system may be shown with reference to fig. 2.
Specifically, the automatic detection system for bridge defects comprises three levels of detection contents when a corresponding method is operated. And for the detection of the whole level of the bridge, the unmanned aerial vehicle automatically plans a route according to the trend of the bridge. A large number of multi-angle images of all parts of the bridge are continuously shot, all parts of the bridge are identified, the positions of all parts are marked according to the positioning information of the unmanned aerial vehicle, and the positions are used as key interest points of second-level route planning. And for component level detection, a detection route is automatically planned according to the fact that the position of the component detected in the last step is taken as an interest point and the shortest route is taken as the principle, and the unmanned aerial vehicle automatically flies according to the route. When the aircraft flies to a component position, images and depth information of the surface of the component are collected through a mounted camera, and diseases are identified in real time. When the disease is identified, the position of the disease is recorded through the positioning information of the unmanned aerial vehicle. For local level detection, aiming at the disease position positioned in the last step, planning a route for inspection of the third level by taking the position as an interest point. When the third-level fine disease detection is carried out, the unmanned aerial vehicle adsorbs to the surface of the structure, acquires fine disease images and disease surface depth information, and directly obtains the geometric information of the diseases through high-dimensional neural network analysis. Meanwhile, a contact type scanning impact echo system carried by the unmanned aerial vehicle carries out continuous knocking test on the disease position to obtain damage information such as crack depth, internal damage and concrete deterioration.
When the automatic detection system for bridge diseases works, the unmanned aerial vehicle comprises three flight states: a normal flight state, a facade adsorption state and a top adsorption state. In a common flight state, the adsorption motor does not work, the lift force motor provides lift force, and the flight mode of the unmanned aerial vehicle is the same as that of a common quad-rotor unmanned aerial vehicle; when facade absorption state, the installation direction through making anterior horizontal absorption motor and flexible supporting wheel be theta with vertical direction and be 5 contained angles, make unmanned aerial vehicle be the state of slightly inclining forward at the fuselage in the twinkling of an eye that contacts the structure facade, therefore in the twinkling of an eye that unmanned aerial vehicle contacted the structure surface, vertical lift motor can produce horizontal component F simultaneously Vx ＝F V sin theta, push away unmanned aerial vehicle tightly and adsorb in structure facade, avoid adsorbing unmanned aerial vehicle platform unstability in the twinkling of an eye. At the moment, the up-down movement of the unmanned aerial vehicle depends on the vertical component force F of the lift motor Vy ＝F V The cos theta is controlled, when the component force is larger than the gravity G, the unmanned aerial vehicle climbs, and otherwise, the unmanned aerial vehicle descends; during the top adsorption state, lift motor provides lift and adsorption affinity, provides the thrust of equal size or differential size by adsorbing the motor and promotes unmanned aerial vehicle and remove. The method can give consideration to the high efficiency and the integrity of detection, and gradually expand the whole bridge structure detection process from three layers of integral, structural member and local in stages. Because the detection range of the whole level and the component level is wide, the detection speed is high, and the detection is relatively rough, the unmanned aerial vehicle carries out scanning detection in a common flight state, and automatic identification of diseases is carried out by utilizing a multi-scale feature fusion deep learning network designed aiming at the scale change of an object in the detection process. For local level detection, the unmanned aerial vehicle adopts an adsorption crawling state to expandAcoustic-wave excited detection of internal damage.
Specifically, the automatic detection system for bridge diseases can firstly analyze key parts of a structure and plan an initial routing inspection route based on an information feedback self-adaptive routing inspection dynamic route planning method, then converts the optimization problem of the initial routing inspection path into a traveler problem in a mathematical graph theory, feeds back all levels of real-time detection results in real time as a key objective function, dynamically and adaptively corrects the unmanned aerial vehicle route by combining a fast extended random tree algorithm to enable all interest points to be connected in series by the shortest path, realizes the unmanned aerial vehicle movement route self-adaptive dynamic planning based on the detection result real-time feedback, and further realizes the fast and efficient detection of the bridge structure diseases.
Specifically, the automatic detection system for the bridge diseases can also be based on a bridge disease automatic mining processing method of a multi-scale deep learning network, and the key structural feature of the automatic detection system is a series-parallel combined structure, namely, three parallel sub-networks of a large scale, a medium scale and a small scale are respectively designed by changing the step length of a convolution kernel for each level of detection in feature extraction, the size of a template frame is determined by dimension clustering in feature prediction, and the target score of each feature is adjusted by utilizing linear regression, so that the feature extraction and prediction of objects with different scales are realized simultaneously.
In one embodiment, the positioning system comprises a GPS positioning system and a visual navigation system, the GPS positioning system provides positioning information when there is a GPS signal, and the visual navigation system provides positioning information automatically when there is no GPS signal.
The visual navigation system can realize unmanned aerial vehicle navigation in a GPS-free environment, wherein a filtering-based loose coupling fusion method is adopted for data fusion of the binocular camera and the inertial measurement unit, the six-degree-of-freedom position and pose information of the unmanned aerial vehicle is solved by adopting vision and inertial navigation respectively, and then the position and pose information is fused by utilizing extended Kalman filtering to obtain optimized six-degree-of-freedom position and pose information.
The present embodiment employs a visual navigation system and a GPS positioning system to cooperatively provide positioning information, wherein the visual navigation system includes a binocular camera, an inertial measurement unit, an optical flow sensor, and an ultrasonic ranging sensor. For bridge detection, the GPS-free environment is mainly the bottom of the bridge. Aiming at the bottom of the bridge, a binocular camera and an inertia measurement unit perform data fusion to provide positioning information, and an ultrasonic sensor is mounted at the top of an unmanned aerial vehicle to measure the distance between the unmanned aerial vehicle and the bottom of the bridge and is used for avoiding obstacles at the top of the unmanned aerial vehicle; the optical flow sensor plays a role in auxiliary positioning in the takeoff phase.
In one example, a framework diagram of a smart drone system may be illustrated with reference to fig. 3. The intelligent unmanned aerial vehicle system comprises a vibration exciter for continuously knocking the surface of the structure and a receiver which is attached to the surface of the structure and rolls to collect an impact echo signal, wherein the vibration exciter is used for continuously knocking the surface of the structure, and the receiver is used for continuously collecting the echo signal while the unmanned aerial vehicle moves and knocks in the process of surface movement of the adsorption structure. For damages such as concrete cracks and internal holes, reflected waves are generated when the exciting waves reach the damaged surface, and the reflected waves are received by a receiver and then converted into a frequency domain through fast Fourier transform, so that the frequency peak value of the reflected waves is determined. The defect depth can be calculated as D ═ β × V p ) V2 f, where β is the form factor, V being determined by the component type p Is the propagation speed of sound wave in the material; f is the calculated dominant frequency.
The automatic detection system for the bridge diseases is a deep innovation and organic integration of a novel automatic unmanned aerial vehicle platform, an artificial intelligence technology, a computer vision technology and a nondestructive inspection technology, can practically and effectively promote the automation and the intelligent degree of bridge detection, is simple and convenient to operate, quick in detection and high in detection result precision, and is particularly suitable for comprehensive quick detection of large-base-number infrastructures.
In an embodiment, the working process of the intelligent drone system may refer to fig. 4, which specifically includes:
at first carry out whole level and detect, because bridge construction is long-line shape, the bridge moves towards or be the straight line or be the curve, after unmanned aerial vehicle takes off, begins from bridge one end, distinguishes the bridge through the camera that carries on unmanned aerial vehicle and moves towards, guides unmanned aerial vehicle to fly to the other end from one end automatically. The unmanned aerial vehicle shoots bridge surface image simultaneously in the flight process, and when flying to certain bridge tower or certain section roof beam body, the object classification of shooting is distinguished automatically to unmanned aerial vehicle to record the unmanned aerial vehicle position this moment simultaneously.
And secondly, detecting the component level, connecting the component positions obtained by the previous step in series, and rapidly planning the unmanned aerial vehicle path by adopting a rapid exploration random tree algorithm. For the bottom of the bridge and the side face of the beam body, the unmanned aerial vehicle keeps a height slightly lower than that of the bridge beam body in flight for shooting video detection; for the bridge tower, the unmanned aerial vehicle makes a tour from top to bottom at the bridge tower measurement and shoots and detects. And in the detection process, the surface of the component is shot at the same time, and whether diseases exist is identified in real time through the acquired video and the lightweight deep learning network. If there is a disease, then the unmanned aerial vehicle position at this moment is recorded.
And then, local disease detection is carried out, and for the position where the disease is detected in the previous step, serial connection is carried out and the shortest detection path is rapidly planned. When arriving a certain disease position, unmanned aerial vehicle is changed into by normal flight and adsorbs the state of crawling, hugs closely and detects on the structure surface. In addition, on one hand, image information is collected through a camera during detection, and analysis is carried out through a multi-scale deep learning network; and on the other hand, a vibration exciter on the unmanned aerial vehicle starts to knock the disease position in a scanning mode, impact echoes are collected, and whether internal damage exists in the disease position or not can be respectively analyzed through echo information analysis, or information such as crack depth is calculated.
In one example, a typical bridge detection embodiment is utilized to illustrate the specific implementation steps of the automatic detection system for bridge diseases.
Step 1: and (5) preparing the system. Firstly, selecting a proper detection initial position, generally selecting the position of one end of a bridge under the bridge for the bridge, flatly placing an unmanned aerial vehicle, and waiting for system self-checking after switching on a power supply. And starting a detection terminal, a positioning terminal and a route planning terminal of the unmanned aerial vehicle through a ground computer which is in wireless communication with the unmanned aerial vehicle. After the unmanned aerial vehicle finishes self-checking, giving an instruction for taking off the unmanned aerial vehicle through the route planning terminal, and starting automatic inspection by the unmanned aerial vehicle.
Step 2: and detecting the whole layer. Unmanned aerial vehicle highly automatic the tour of about 2m in the roof beam body lower part detects once in the testing process, and whole testing process unmanned aerial vehicle keeps the certain distance with the bridge, and consequently the camera visual field is big, detects fastly, and for thick detection, unmanned aerial vehicle's location is provided by GPS, and whole process is accomplished fastly, exports the positional information of each component of bridge at last.
And step 3: and detecting the component level. And rapidly planning the shortest route according to the position information of the last step, and carrying out second-level component detection by the unmanned aerial vehicle according to the route. To specific component detection, the detection process of the unmanned aerial vehicle can be divided into three states: for the detection of the bridge tower, the positioning of the unmanned aerial vehicle is still provided by the GPS because the GPS of the unmanned aerial vehicle is not shielded by the bridge; for bridge body detection, an unmanned aerial vehicle mainly works at the bottom of a bridge, and detection states are divided into two states, one state is that the unmanned aerial vehicle just changes from GPS positioning to visual positioning, and the visual positioning needs to be initialized briefly, so that the positioning of the unmanned aerial vehicle is provided by an optical flow sensor at the moment; when unmanned aerial vehicle is close to the bridge bottom, unmanned aerial vehicle's location is resolved by two mesh vision and inertial navigation and is obtained, and the ultrasonic sensor at unmanned aerial vehicle top provides the range finding simultaneously and keeps away the barrier function. The detection of the second-level unmanned aerial vehicle on the diseases is also completed by video analysis of the camera, and the analysis method is obtained by real-time identification of a lightweight deep learning model embedded in an onboard computer of the unmanned aerial vehicle. The detection result of the second level is the type of each disease and the position of the disease.
And 4, step 4: and (5) detecting local diseases. To the disease position that last step obtained, carry out fast path planning again, the purpose that local detection is detailed disease detection, whenever reach a disease position when unmanned aerial vehicle, unmanned aerial vehicle converts the absorption formula into from ordinary flight and crawls, and unmanned aerial vehicle hugs closely and detects in bridge bottom or pier side. The detection process adopts a high-resolution camera and a designed scanning impact echo system, the data of the high-resolution camera is used for surface disease detection, and the impact echo data is used for internal damage detection.
And 5: and analyzing surface diseases and internal damage. Step 2 and step 3 obtain the complete disease identification result and the disease position information of the bridge surface, and step 4 obtains the image with detailed diseases and the surface depthInformation and shock echo information, and therefore the detailed disease information collected in step 4 is then analyzed. And analyzing the acquired image data by adopting a multi-scale deep learning network to directly obtain the length, width and area information of the segmented diseases. Converting the echo in the process of scanning the impact echo into a frequency domain by using fast Fourier transform, and determining the frequency peak value of the echo, the depth of the internal defect can be calculated as D ═ beta x V p )/2f。
The steps are adopted to detect the bridge, the required time is far shorter than that of the traditional manual detection, and compared with the manual detection, the method has the advantages that the disease classification result goodness of fit is over 90 percent, but the method can quickly obtain the geometric information and the internal damage of the disease, and the qualitative result is more accurate and effective than that of the traditional manual detection. The comprehensive evaluation shows that the method has good accuracy, practicability and advancement and has wide application prospect in automatic detection of engineering structure diseases.
In one example, fig. 5 is a schematic diagram of a three-hierarchy disease identification network based on multi-scale deep learning.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
It should be noted that the terms "first \ second \ third" referred to in the embodiments of the present application are only used for distinguishing similar objects, and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may interchange a specific order or sequence when allowed. It should be understood that "first \ second \ third" distinct objects may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented in an order other than those illustrated or described herein.
The terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that comprises a list of steps or modules is not limited to only those steps or modules recited, but may alternatively include other steps or modules not recited, or that are inherent to such process, method, product, or device.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (5)
1. An automatic detection method for bridge diseases is characterized in that bridge detection is divided into three layers of integral bridge, dispersed components and local diseases, a multi-mode unmanned aerial vehicle with flying and adsorption crawling is used as a platform for detection, the unmanned aerial vehicle is guided to automatically expand inspection by a dynamic inspection path automatic planning method based on real-time detection result feedback in the detection process of each layer, and after inspection is finished, disease characteristics in obtained information are analyzed and excavated through a multi-layer deep learning network corresponding to the scale of the three-layer detection characteristics;
the designed multi-scale deep learning network is a series-parallel combined network structure, and the whole network is divided into an independent feature extraction layer and a shared feature prediction layer through series-parallel connection; the system comprises a feature extraction layer, a plurality of sets of convolution layers, a batch normalization layer, an activation layer and a residual layer, wherein the feature extraction layer respectively designs three parallel feature extraction sub-networks according to the overall dimension, the component dimension and the local disease dimension of a bridge, the structure of a single sub-network is similar to that of a common feature extraction network and comprises the sets of convolution layers, the batch normalization layer, the activation layer and the residual layer, for large-scale input, the residual layers are more, the deeper the network is, and the larger the down-sampling multiplying power is; for the characteristic prediction layer, firstly, a certain number of candidate frames with different length-width ratios are obtained through clustering analysis, then, intensive sampling is uniformly carried out on different positions of a characteristic diagram, and classification and regression are directly carried out;
the designed multi-scale deep learning network is a lightweight network, and parameters and complexity of the model are reduced by adopting a depth-divisible volume and a chain architecture, so that the model can directly run in an onboard computer carried by the unmanned aerial vehicle in real time, and structural surface images shot by an unmanned aerial vehicle camera are processed in real time;
for the whole level detection, the core is to abstract the trend of a bridge member according to the geometric shape of the bridge, judge the video shot by a camera carried by the unmanned aerial vehicle by using a classification neural network so as to identify the pointing direction and the spatial position of the unmanned aerial vehicle relative to the bridge member at each moment, further guide the unmanned aerial vehicle to automatically inspect according to the trend of the member, identify each member of the bridge and record the spatial position, and take the spatial position as an interest point; for component and local level detection, the core of the method is to convert the optimization problem of the optimal route for unmanned aerial vehicle routing inspection into a mathematical TSP problem according to the interest points detected in the previous level, namely, firstly, a state transfer function go (S, init) is established according to the time loss and the Manhattan distance of the interest points, wherein the min { go (S-i, i) + dp [ i ] [ init ] }, and then a Hamilton loop with the minimum weight is searched through a fast search random tree algorithm, so that the total distance go (S, init) traversing all the interest points S from the first init point is minimum, and the unmanned aerial vehicle routing inspection route is shortest and the efficiency is highest.
2. The system for implementing the automatic detection method for the bridge diseases according to claim 1 is characterized in that an unmanned aerial vehicle system for implementing detection work has two detection modes of a flight mode and an adsorption crawling mode, and the detection is carried out by a flight model in overall and component level detection to quickly acquire a macroscopic disease area on the surface of a structure; in local level detection, an unmanned aerial vehicle is directly adsorbed on the surface of the structure, so that the obtained image information has the characteristics of short distance, small visual field and high precision, knocking detection is carried out on the position with the disease, and the crack depth, the internal damage and the concrete degradation of the structure are judged through the obtained sound wave echo;
the unmanned aerial vehicle system comprises a flight module, an adsorption module and a detection module;
the flight module comprises a lift system in an X-type layout with the equal space of 550mm and a navigation control system based on multi-sensor fusion, and is used for realizing six-degree-of-freedom flight of the unmanned aerial vehicle;
the navigation control system based on multi-sensor fusion comprises a GPS positioning system and a vision-inertial navigation positioning system, wherein the GPS positioning system provides positioning information when a GPS signal exists, and the vision navigation system provides positioning information automatically when no GPS signal exists; the data fusion of the binocular camera and the inertial measurement unit in the visual navigation system adopts a filtering-based loose coupling fusion method, firstly, the vision and inertial navigation are respectively adopted to solve the six-degree-of-freedom pose information of the unmanned aerial vehicle, and then the pose information is fused by using extended Kalman filtering to obtain the optimized six-degree-of-freedom pose information.
3. The system of claim 2, wherein the adsorption module comprises a horizontal adsorption motor and a specially designed forward-leaning frame, when the horizontal adsorption motor and the flexible supporting wheel are installed in the front part, the horizontal adsorption motor and the flexible supporting wheel are arranged in an included angle of theta-5 degrees with the vertical direction when the horizontal adsorption motor and the flexible supporting wheel are in the vertical surface adsorption state, so that the unmanned aerial vehicle body is in a slightly forward-leaning state at the moment of contacting the vertical surface of the structure, and the vertical lift motor can simultaneously generate a horizontal component force F at the moment of contacting the surface of the structure Vx ＝F V sin theta, tightly pushing the unmanned aerial vehicle and adsorbing the unmanned aerial vehicle on the vertical surface of the structure, so as to avoid instability of the unmanned aerial vehicle platform at the moment of adsorption; vertical component force F depending on lift motor for up-and-down movement of unmanned aerial vehicle Vy ＝F V The cos theta is controlled, when the component force is larger than the gravity G, the unmanned aerial vehicle climbs, and otherwise, the unmanned aerial vehicle descends; during the top adsorption state, lift motor provides lift and adsorption affinity, provides the thrust of equal size or differential size by adsorbing the motor and promotes unmanned aerial vehicle and remove.
4. An automatic detection method for bridge diseases is characterized by comprising the following steps:
s10, carrying out overall hierarchical coarse detection, automatically planning a route by the intelligent unmanned aerial vehicle system according to the trend of the bridge, identifying components of each part of the bridge by continuously shooting images of each part of the bridge at multiple angles, marking the position of each component according to the positioning information of the unmanned aerial vehicle, and taking the position as a key interest point for second-level route planning;
s20, performing hierarchical detection on the components, planning a detection route by taking the position of the component detected in the previous step as a key interest point and taking the shortest route as a principle, flying the unmanned aerial vehicle according to the detection route, acquiring image information of a small view field on the surface of the component through a carried camera when the unmanned aerial vehicle flies to one detection route, and detecting diseases according to the acquired images; when a disease is detected, recording the position of the detected disease through the positioning information of the unmanned aerial vehicle;
s30, local level precise detection, planning a third level routing inspection route by taking the disease position detected in the previous step as an interest point, and during the third level fine disease detection, adsorbing the structure surface at the disease position by an unmanned aerial vehicle, collecting a disease image and the internal damage information of the disease surface, and directly obtaining the geometric information of the disease through the analysis of a recurrent neural network;
the route planning method in the steps S20 and S30 comprises the following steps: firstly, establishing a state transfer function go (S, init) min { go (S-i, i) + dp [ i ] [ init ] } according to the time loss and the Manhattan distance of the interest points, and then searching a Hamilton loop with the minimum weight through a fast search random tree algorithm to ensure that the total distance go (S, init) from the first init point to traverse all the interest points S is minimum, so that the unmanned aerial vehicle routing route is shortest and the efficiency is highest;
the internal damage detection method is a detection method based on frequency conversion pulse excitation and sound wave feature separation, an unmanned aerial vehicle moves and knocks while moving on the surface of an adsorption structure, echo signals are continuously collected, for the internal damage of concrete, a reflected wave is generated when the excited wave reaches the damaged surface, the reflected wave is received by a receiver and then converted into a frequency domain through fast Fourier transform, the frequency peak value of the echo is determined, and the defect depth is calculated as D ═ beta x V p ) V2 f, where β is the form factor, V being determined by the component type p Is the propagation velocity of the acoustic wave in the material, and f is the calculated primary frequency.
5. The automatic detection method according to claim 4, characterized in that for the detection of the whole hierarchy, the trend of the bridge part component is abstracted according to the geometric shape of the bridge, the classification neural network is used for judging the video shot by the unmanned aerial vehicle carrying camera so as to identify the pointing direction and the spatial position of the unmanned aerial vehicle relative to the part component at each moment, further guiding the unmanned aerial vehicle to automatically patrol according to the trend of the part component, simultaneously identifying each part component of the bridge and recording the position of each part component, and taking the position of each part component as a key interest point for the hierarchical route planning of the second hierarchy component.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010842289.7A CN112098326B (en) | 2020-08-20 | 2020-08-20 | Automatic detection method and system for bridge diseases |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010842289.7A CN112098326B (en) | 2020-08-20 | 2020-08-20 | Automatic detection method and system for bridge diseases |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112098326A CN112098326A (en) | 2020-12-18 |
| CN112098326B true CN112098326B (en) | 2022-09-30 |
Family
ID=73753080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010842289.7A Active CN112098326B (en) | 2020-08-20 | 2020-08-20 | Automatic detection method and system for bridge diseases |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112098326B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113570915A (en) * | 2021-07-08 | 2021-10-29 | 云南国土资源职业学院 | Think removal explanation device of political affairs classroom teaching usefulness |
| CN113267139B (en) * | 2021-07-19 | 2021-10-29 | 江苏中科云控智能工业装备有限公司 | Die casting deformation amount detection system with big data analysis |
| CN113791140B (en) * | 2021-11-18 | 2022-02-25 | 湖南大学 | Bridge bottom interior nondestructive testing method and system based on local vibration response |
| CN114657874B (en) * | 2022-04-08 | 2022-11-29 | 哈尔滨工业大学 | Intelligent inspection robot for bridge structure diseases |
| CN114494891B (en) * | 2022-04-15 | 2022-07-22 | 中国科学院微电子研究所 | Hazardous article identification device and method based on multi-scale parallel detection |
| CN114612473B (en) * | 2022-05-11 | 2022-08-16 | 深圳联和智慧科技有限公司 | Bridge inspection processing method and system based on unmanned aerial vehicle and cloud platform |
| CN115096268B (en) * | 2022-06-17 | 2023-06-30 | 西南交通大学 | Bridge damage depth detection method based on unmanned aerial vehicle aerial photography and ultrasonic detection |
| CN115639209B (en) * | 2022-12-26 | 2023-03-07 | 兰州交通大学 | Intelligent detection system and method for fatigue cracks of steel box girder |
| CN117067226A (en) * | 2023-08-17 | 2023-11-17 | 兰州交通大学 | Steel bridge rust detection robot and rust detection method |
| CN117699085B (en) * | 2024-02-04 | 2024-04-19 | 安徽省交规院工程智慧养护科技有限公司 | Detection equipment and method for bridge disease detection |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106774384A (en) * | 2016-12-05 | 2017-05-31 | 王源浩 | A kind of bridge machinery intelligent barrier avoiding robot |
| CN108182402A (en) * | 2017-12-27 | 2018-06-19 | 同济大学 | A kind of steel box-girder method for inspecting for using acquisition robot and deep learning algorithm |
| CN109282808A (en) * | 2018-11-23 | 2019-01-29 | 重庆交通大学 | Unmanned plane and Multi-sensor Fusion localization method for the detection of bridge Cut-fill |
| CN109901624A (en) * | 2019-04-11 | 2019-06-18 | 株洲时代电子技术有限公司 | A kind of bridge method for inspecting |
| CN110703802A (en) * | 2019-11-04 | 2020-01-17 | 中国科学院自动化研究所 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
-
2020
- 2020-08-20 CN CN202010842289.7A patent/CN112098326B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106774384A (en) * | 2016-12-05 | 2017-05-31 | 王源浩 | A kind of bridge machinery intelligent barrier avoiding robot |
| CN108182402A (en) * | 2017-12-27 | 2018-06-19 | 同济大学 | A kind of steel box-girder method for inspecting for using acquisition robot and deep learning algorithm |
| CN109282808A (en) * | 2018-11-23 | 2019-01-29 | 重庆交通大学 | Unmanned plane and Multi-sensor Fusion localization method for the detection of bridge Cut-fill |
| CN109901624A (en) * | 2019-04-11 | 2019-06-18 | 株洲时代电子技术有限公司 | A kind of bridge method for inspecting |
| CN110703802A (en) * | 2019-11-04 | 2020-01-17 | 中国科学院自动化研究所 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
Non-Patent Citations (2)
| Title |
|---|
| 基于无人机视觉的桥梁裂缝检测;陶晓力等;《计算机技术与发展》;20171115(第03期);全文 * |
| 无人机在路桥病害检测中的设计与实现;陈显龙等;《测绘通报》;20160425(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112098326A (en) | 2020-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112098326B (en) | Automatic detection method and system for bridge diseases | |
| Jiang et al. | Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system | |
| Zhou et al. | Robust real-time UAV based power line detection and tracking | |
| CN108593656B (en) | Structure detection method and device and unmanned aerial vehicle system for structure detection | |
| Li et al. | Automatic bridge crack detection using Unmanned aerial vehicle and Faster R-CNN | |
| CN108051450B (en) | Bridge health detection system and method based on unmanned aerial vehicle | |
| DK2702382T3 (en) | METHOD AND SYSTEM FOR INSPECTION OF A SURFACE ERROR FOR MATERIAL ERROR | |
| CN109780452A (en) | Gas based on laser telemetry technology leaks unmanned plane inspection retrieving concentration method | |
| CN105404844A (en) | Road boundary detection method based on multi-line laser radar | |
| CN109829908B (en) | Binocular image-based method and device for detecting safety distance of ground object below power line | |
| CN102253381A (en) | System and method for automatically detecting foreign object debris (FOD) on airfield runways | |
| Kalaitzakis et al. | Dynamic structural health monitoring using a DIC-enabled drone | |
| EP3850456B1 (en) | Control and navigation systems, pose optimisation, mapping, and localisation techniques | |
| CN111638220A (en) | Bridge crack detection system of interactive intelligent unmanned aerial vehicle group based on 5G | |
| CN110104167A (en) | A kind of automation search and rescue UAV system and control method using infrared thermal imaging sensor | |
| CN113985907A (en) | Tree barrier risk prediction and optimization method based on multi-load data of unmanned aerial vehicle | |
| Minghui et al. | Deep learning enabled localization for UAV autolanding | |
| Poorghasem et al. | Review of robot-based automated measurement of vibration for civil engineering structures | |
| Liu et al. | Framework for automated UAV-based inspection of external building façades | |
| CN113776467A (en) | Highway ramp road surface technical condition detecting system based on unmanned aerial vehicle platform | |
| CN116188470B (en) | Unmanned aerial vehicle aerial photographing identification-based fault positioning method and system | |
| KR102191109B1 (en) | Autonomous unmanned aerial vehicle and control method in the same | |
| CN117288094A (en) | Real-time positioning system of heading machine based on laser sensor | |
| Mitra et al. | A UAV Payload for Real-time Inspection of Highway Ancillary Structures | |
| EP4091139A1 (en) | Inspection device for inspecting a building or structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |