WO2022193420A1 - 近水桥梁多类型病害智能检测方法与无人船设备 - Google Patents

近水桥梁多类型病害智能检测方法与无人船设备 Download PDF

Info

Publication number
WO2022193420A1
WO2022193420A1 PCT/CN2021/092393 CN2021092393W WO2022193420A1 WO 2022193420 A1 WO2022193420 A1 WO 2022193420A1 CN 2021092393 W CN2021092393 W CN 2021092393W WO 2022193420 A1 WO2022193420 A1 WO 2022193420A1
Authority
WO
WIPO (PCT)
Prior art keywords
module
attention
network
detection
pam
Prior art date
Application number
PCT/CN2021/092393
Other languages
English (en)
French (fr)
Inventor
张建
何至立
蒋赏
Original Assignee
东南大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 东南大学 filed Critical 东南大学
Priority to US17/755,086 priority Critical patent/US20230351573A1/en
Publication of WO2022193420A1 publication Critical patent/WO2022193420A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B1/00—Hydrodynamic or hydrostatic features of hulls or of hydrofoils
    • B63B1/02—Hydrodynamic or hydrostatic features of hulls or of hydrofoils deriving lift mainly from water displacement
    • B63B1/10—Hydrodynamic or hydrostatic features of hulls or of hydrofoils deriving lift mainly from water displacement with multiple hulls
    • B63B1/12—Hydrodynamic or hydrostatic features of hulls or of hydrofoils deriving lift mainly from water displacement with multiple hulls the hulls being interconnected rigidly
    • B63B1/125—Hydrodynamic or hydrostatic features of hulls or of hydrofoils deriving lift mainly from water displacement with multiple hulls the hulls being interconnected rigidly comprising more than two hulls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B35/00—Vessels or similar floating structures specially adapted for specific purposes and not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B45/00—Arrangements or adaptations of signalling or lighting devices
    • B63B45/04—Arrangements or adaptations of signalling or lighting devices the devices being intended to indicate the vessel or parts thereof
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B79/00—Monitoring properties or operating parameters of vessels in operation
    • B63B79/40—Monitoring properties or operating parameters of vessels in operation for controlling the operation of vessels, e.g. monitoring their speed, routing or maintenance schedules
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
    • G01M5/0008—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings of bridges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
    • G01M5/0033—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings by determining damage, crack or wear
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
    • G01M5/0075—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings by means of external apparatus, e.g. test benches or portable test systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
    • G01M5/0091—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings by using electromagnetic excitation or detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B35/00—Vessels or similar floating structures specially adapted for specific purposes and not otherwise provided for
    • B63B2035/006—Unmanned surface vessels, e.g. remotely controlled
    • B63B2035/008—Unmanned surface vessels, e.g. remotely controlled remotely controlled
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B63—SHIPS OR OTHER WATERBORNE VESSELS; RELATED EQUIPMENT
    • B63B—SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING 
    • B63B2211/00—Applications
    • B63B2211/02—Oceanography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30184—Infrastructure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means

Definitions

  • the invention belongs to the field of structural health detection in civil engineering, and in particular relates to an intelligent detection method for multi-type diseases of near-water bridges and unmanned ship equipment.
  • the intelligent detection method is represented by deep learning technology, which has brought revolutionary solutions to many industries, such as medicine and health, aerospace and materials science, etc.
  • the learned medical image segmentation method discloses a material property prediction method based on deep learning.
  • the use of deep learning techniques for intellisense of structural diseases is attracting more and more attention.
  • researchers apply deep learning methods to the detection of different diseases and different infrastructures.
  • the patent document with the publication number CN112171692A discloses a flying adsorption robot suitable for intelligent detection of bridge deflection;
  • the patent document with the publication number CN111413353A discloses an intelligent mobile comprehensive detection equipment for tunnel lining diseases;
  • the publication number is The patent document CN111021244A discloses an orthotropic steel bridge deck fatigue crack detection robot;
  • the patent document with the publication number of CN109978847A discloses a cable-robot-based method for identifying the disease of the noose.
  • the current intelligent detection method is mainly based on the Anchor-based method, that is, a large number of a priori boxes need to be pre-set, that is, anchor boxes, so it is named Anchor-based method.
  • Anchor-based method a large number of a priori boxes need to be pre-set, that is, anchor boxes, so it is named Anchor-based method.
  • the patent document with the publication number CN111062437A discloses a bridge disease target detection model based on the Faster R-CNN model
  • the patent document with the publication number CN111310558A also discloses a road damage extraction method based on the Faster R-CNN model.
  • the patent document whose publication number is CN111127399A discloses a method for detecting bridge pier diseases based on the YOLOv3 model.
  • Anchor-based methods Both the Faster R-CNN model and the YOLO series of models are very classic Anchor-based methods.
  • the first prominent problem of Anchor-based methods is that the effect of the algorithm will be affected by the pre-set prior box. When dealing with features with complex shapes, multiple aspect ratios and multiple sizes, the size and aspect ratio of the prior box may be too different from the target, which will reduce the recall rate of the prediction results. Therefore, in order to improve the detection effect, a large number of prior frames are often preset. This also brings about the second outstanding problem of the Anchor-based method. A large number of a priori boxes will introduce a large number of hyperparameters and design choices, which will make the model very complex, with a large amount of calculation, and the calculation efficiency is often not high. .
  • the detectable area of intelligent equipment is still very limited, mainly facing the area that is easy to detect such as the outer surface of the structure.
  • the detection method discloses a method for detecting apparent diseases of bridges based on UAV.
  • UAV systems are difficult to work in relatively closed spaces, such as the bottom area of a large number of small and medium bridges, where the headroom is low, and the situation is complex, and artificial and intelligent detection equipment is often helpless.
  • the present invention discloses an intelligent detection method for multi-type diseases of near-water bridges and unmanned ship equipment, which are suitable for automatic and intelligent detection of diseases at the bottom of small and medium bridges.
  • the proposed solution includes intelligent algorithms and hardware equipment. It can ensure the detection accuracy while taking into account the detection speed, and has good generalization ability and applicability to complex engineering environments.
  • the intelligent detection method for multi-type diseases of bridges near water includes the following components:
  • Component 1 Intelligent detection algorithm: CenWholeNet, an infrastructure disease target detection network based on deep learning;
  • Component 2 Embed the parallel attention module PAM into the target detection network CenWholeNet, the parallel attention module includes two sub-modules, a spatial attention sub-module and a channel attention sub-module;
  • Component three intelligent detection equipment: an unmanned ship system based on lidar navigation, the unmanned ship system includes four modules, a hull module, a video acquisition module, a lidar navigation module and a ground station module.
  • the infrastructure disease target detection network CenWholeNet described in the first component includes the following steps: step 1: backbone network: the backbone network is used to extract image features; step 2: detector: the detector converts the extracted image features In order to calculate the required tensor form, the loss function is used for optimization; Step 3: Result output: The result output is to convert the tensor into a bounding box to realize the prediction result output of target detection.
  • step 1 of the infrastructure disease target detection network CenWholeNet described in the first component the specific method of the backbone network is as follows:
  • W is the width of the image
  • H is the height of the image
  • 3 is the number of channels of the image, that is, three RGB channels
  • the features of the input image P are extracted through the backbone network; this patent recommends using two influential convolutional neural networks.
  • step 2 of the infrastructure disease target detection network CenWholeNet described in component 1 the specific method of the detector is as follows:
  • the detector is the core of CenWholeNet, which converts the features extracted by the backbone network into an output set consisting of 4 tensors
  • the model can obtain the rough width and height of each prediction box
  • the 2 of the first dimension represents the offset of the key point (x, y) in the W and H directions; correspondingly, the model will give a prediction tensor We use smooth L1 Loss to train the offset loss:
  • the inclination angle ⁇ k of the line connecting the two corner points can be calculated by the following formula:
  • the model predicts the output of C+6, which makes up the set They will also share the weights of the network; the overall loss function of the network can be defined by:
  • step 3 of the infrastructure disease target detection network CenWholeNet described in the first component the specific steps of the result output are as follows:
  • the work to be done in the result output part is to extract the heatmap tensor from the predicted Extract the possible center keypoint coordinates from and information in to get the predicted bounding box; obviously, The larger the value of , the more likely it is the center point; for category c, if the point p cxy satisfies the following formula, it can be considered that p cxy is an alternative center point;
  • NMS non-maximum suppression
  • attention plays a very important role in human perception.
  • human eyes or ears and other organs acquire information, they tend to focus on more interesting targets and improve their attention; while suppressing uninteresting targets, Reduce its attention;
  • attention mechanism by embedding attention modules in neural networks, increase the weight of feature tensors in meaningful regions , reducing the weights of areas such as meaningless backgrounds, thereby improving the performance of the network;
  • this patent proposes a lightweight, plug-and-play parallel attention module PAM, and experiments have verified that PAM can significantly improve the expression of neural networks.
  • PAM considers the attention of two dimensions of feature map, spatial attention and channel attention, and combines them in parallel;
  • the given input feature map is where C, H, and W represent the channel, height, and width, respectively; first, the transformation is implemented by the spatial attention submodule Then, the transformation is implemented by the channel attention submodule The final output feature map
  • the transformation mainly includes convolution, maximum pooling operation, mean pooling operation and ReLU function, etc.
  • the overall calculation process is as follows:
  • PConv point-wise convolution
  • ⁇ represents the ReLU activation function
  • the size of the convolution kernel of PConv1 is C/r ⁇ C ⁇ 1 ⁇ 1
  • the size of the convolution kernel of the inverse transform PConv2 is C ⁇ C/r ⁇ 1 ⁇ 1
  • the ratio r is recommended to be 16 , other scaling ratios are optional; it should be noted that the channel attention weights will be replicated along the width and height directions;
  • Our proposed PAM is a plug-and-play module, which ensures strict consistency of input tensors and output tensors at the dimension level; therefore, it can theoretically be embedded in any position of any convolutional neural network model as a supplementary module; this
  • the patent gives two recommended solutions for PAM embedding Hourglass and ResNet.
  • the ResNet network the PAM is embedded in the residual block after the batch normalization layer, before the residual connection, and the same in each residual block. Operation; for the Hourglass network, it is divided into two parts: downsampling and upsampling.
  • the downsampling part embeds the PAM between the residual blocks as a transition module
  • the upsampling part embeds the PAM before the residual connection. Details see Attachment.
  • the unmanned ship system includes four modules, hull module, video acquisition module, lidar navigation module and ground station module.
  • the modules cooperate with each other and work together;
  • the hull module includes a trimaran and a power system; the trimaran design can make the ship more stable, designed to resist 6-level wind and waves, and the effective remote control distance is 500 meters, which can basically adapt to most engineering application scenarios; the size of the hull is 75 ⁇ 47 ⁇ 28 cm, easy to transport; the unmanned ship has an effective load of 5kg, and can be equipped with multiple scientific instruments; in addition, the unmanned ship has the function of constant speed cruise, reducing the control burden of personnel;
  • the video capture module is composed of a three-axis camera pan/tilt, a fixed front camera, and a fill light; the three-axis camera pan/tilt supports 10x optical zoom, auto focus, photography and 60FPS video recording; this can meet the needs of different scales and different locations.
  • the fixed front camera can easily determine the hull posture; the picture can be transmitted back to the ground station in real time through the wireless image transmission device, on the one hand, the disease can be identified, and on the other hand, it can assist in the control of the USV; in order to cope with light such as the bottom of small and medium bridges Insufficient working environment, we installed a controllable LED fill light board with 180 high-brightness LED lamp beads; 3D printed a gimbal carrying the LED fill light board, which can meet the needs of multi-angle fill light; Equipped with fixed front-view LED lamp beads to provide light source support for the front-view camera;
  • the lidar navigation module includes lidar, mini computer, a set of transmission system and control system; lidar can perform 360° omnidirectional scanning; after it is connected with the mini computer, it can perform real-time mapping of the surrounding environment of the unmanned ship; through wireless Image transmission, the information of the surrounding scene can be transmitted back to the ground station in real time, so as to realize the lidar navigation of the unmanned ship; based on the lidar navigation, the unmanned ship no longer needs GPS positioning.
  • the wireless transmission system supports real-time transmission of 1080P video, and the maximum transmission distance can reach 10 kilometers; redundant transmission is adopted to ensure the link stability and strong anti-interference;
  • the control system consists of wireless image transmission equipment, Pixhawk 2.4.8 Composed of flight control and SKYDROID T12 receiver; through flight control and receiver, we can effectively control the equipment on board;
  • the ground station module includes two remote controllers and many display devices; the main remote controller is used to control the unmanned ship, the secondary remote controller is used to control the shipborne scientific equipment, and the display device is used to monitor the real-time information returned by the camera and lidar; in practical engineering In the detection, the computer is an optional device. On the one hand, it can display the picture in real time, and on the other hand, it can also process the image in real time to identify the disease; the devices cooperate with each other to realize the intelligent disease detection without GPS signal.
  • the present invention is the first application of Anchor-free target detection algorithm in the field of structural diseases.
  • the detection results of the traditional Anchor-based method will be affected by the setting of the a priori box (that is, the anchor boxes), which leads to this algorithm to deal with structural diseases with complex shapes, multiple sizes, and multiple types.
  • the aspect ratio is characterized (for example, the aspect ratio of the steel bar may be large, and the aspect ratio of the peeling may be small)
  • the size and aspect ratio of the preset a priori frame will be very different from the target, which will affect the detection result.
  • the recall rate is low.
  • a large number of a priori frames are often preset. This introduces many hyperparameters and design choices.
  • the method proposed in the present invention abandons the complex a priori frame setting, directly predicts key points and related vectors (ie information such as width, height, etc.), and composes them into a detection frame.
  • the method of the invention is simpler, direct and effective, solves the problem fundamentally, and is more suitable for the detection of engineering structure diseases with complex features.
  • the present invention proposes a novel and lightweight attention module by considering the gain effect of the attention mechanism on the expressive ability of the neural network model.
  • the experimental results show that the method proposed by the present invention is superior to multiple neural network models with extensive influence, and achieves a comprehensive and better effect in the two dimensions of efficiency and accuracy.
  • the proposed attention module can also bring general gains to different neural network models at the expense of negligible computation.
  • the present invention proposes an unmanned ship solution that does not rely on GPS signals to detect diseases at the bottom of small and medium bridges. Due to the constraints of design and performance, the current testing equipment is often helpless when testing the bottom of a large number of small and medium bridges. Taking drones as an example, their flight often requires a wider space free of interference and requires GPS-assisted positioning. However, in the area at the bottom of small and medium bridges with very low clearance, urban underground culverts and sewers, etc., the space is relatively closed, the GPS signal is often very weak, and the internal situation is very complicated. There are risks such as signal loss and collision damage when the drone flies in.
  • the present invention takes the lead in proposing a highly robust unmanned ship system suitable for disease detection in relatively closed areas.
  • the experimental results show that while improving the detection efficiency, the system can reduce the safety risk and detection difficulty of engineers and save a lot of manpower cost, has strong engineering applicability and broad application prospects.
  • the system proposed by the present invention is not only suitable for the bottom of medium and small bridges, but also has great application potential in engineering scenarios such as urban underground culverts and sewers.
  • FIG. 2 is a schematic diagram of the CenWholeNet network proposed by the present invention.
  • FIG. 3 is a detailed diagram of the attention module PAM proposed by the present invention.
  • FIG. 4 is a schematic diagram of the architecture of the unmanned ship system proposed by the present invention.
  • FIG. 5 is a schematic diagram of the polar coordinate supplementary information proposed by the present invention.
  • Fig. 6 the proposed PAM embedded ResNet network scheme diagram of the present invention.
  • FIG. 7 PAM embedded Hourglass network scheme diagram proposed by the present invention.
  • FIG. 8 is a schematic diagram of the application of the method proposed by the present invention in a bridge group
  • FIG. 9 is a schematic diagram of the real-time mapping of the unmanned ship equipment proposed by the present invention.
  • Figure 10 is a schematic diagram of the detection results of the method proposed by the present invention.
  • 11 is a comparison table of the detection results between the algorithm framework proposed by the present invention and other advanced target detection algorithms
  • Fig. 12 The algorithm framework proposed by the present invention is compared with the training process of other advanced target detection algorithms.
  • Part 1 Intelligent detection algorithm: CenWholeNet, a deep learning-based infrastructure disease target detection network, the network details are shown in Figure 2.
  • Part 2 Embed the parallel attention module PAM into the target detection network CenWholeNet.
  • the parallel attention module includes two sub-modules, a spatial attention sub-module and a channel attention sub-module. The specific process is shown in Figure 3.
  • Component three intelligent detection equipment: an unmanned ship system based on lidar navigation, the unmanned ship system includes four modules, a hull module, a video acquisition module, a lidar navigation module and a ground station module.
  • the unmanned ship system architecture scheme is shown in Figure 4.
  • the infrastructure disease target detection network CenWholeNet described in the first component includes the following steps: Step 1: Backbone network: The backbone network is used to extract image features; Step 2: Detector: The detector converts the extracted image features into computational The required tensor form is optimized by the loss function; Step 3: Result output: The result output is to convert the tensor into a bounding box to realize the prediction result output of target detection.
  • step 1 of the infrastructure disease target detection network CenWholeNet described in the first component the specific method of the backbone network is as follows:
  • W is the width of the image
  • H is the height of the image
  • 3 represents the number of channels of the image, that is, three channels of RGB.
  • the features of the input image P are extracted through the backbone network.
  • This patent recommends using two influential convolutional neural network models: Hourglass network and deep residual network ResNet, which are two very classic fully convolutional encoder-decoder networks.
  • ResNet deep residual network
  • step 2 of the infrastructure disease target detection network CenWholeNet described in component 1 the specific method of the detector is as follows:
  • the detector is the core of CenWholeNet, which converts the features extracted by the backbone network into an output set consisting of 4 tensors
  • H c, x, y represents the value of H at the position (c, x, y), that is, the probability that this position is the center point.
  • focal loss as a metric to measure and the distance between H, that is
  • N is the number of all central keypoints
  • L Heat the neural network model can better predict the location of the center point of the target.
  • the model can obtain the rough width and height of each prediction box.
  • 2 in the first dimension represents the offset of the keypoint (x, y) in both W and H directions.
  • the model will give a prediction tensor We use smooth L1 Loss to train the offset loss:
  • the inclination angle ⁇ k of the line connecting the two corner points can be calculated by the following formula:
  • the overall loss function of the network can be defined by:
  • step 3 of the infrastructure disease target detection network CenWholeNet described in the first component the specific steps of the result output are as follows:
  • the work to be done in the result output part is to extract the heatmap tensor from the predicted Extract the possible center keypoint coordinates from and information in to get the predicted bounding box. Obviously, The larger the value of , the more likely it is the center point. For category c, if the point p cxy satisfies the following formula, it can be considered that p cxy is an alternative center point.
  • NMS Non-Maximum Suppression
  • attention plays a very important role in human perception.
  • organs such as the human eye or human ear acquire information, they tend to focus on more interesting targets and improve their attention. And suppress uninteresting targets, reducing their attention.
  • attention mechanism by embedding attention modules in neural networks, the weight of feature tensors in meaningful regions is increased, and the weights of insignificant regions are reduced. The weights of regions such as the background can improve the performance of the network.
  • This patent proposes a lightweight, plug-and-play parallel attention module PAM, and experiments verify that PAM can significantly improve the expressiveness of neural networks.
  • PAM considers two dimensions of feature map attention, spatial attention and channel attention, and combines them in a parallel manner.
  • the given input feature map is where C, H and W represent the channel, height and width, respectively.
  • the transformation is implemented by the spatial attention submodule
  • the transformation is implemented by the channel attention submodule
  • the final output feature map The transformation mainly includes convolution, max pooling operation, mean pooling operation and ReLU function, etc.
  • the overall calculation process is as follows:
  • the spatial attention submodule emphasizes "where" to improve attention, focusing on the locations of regions of interest (ROIs).
  • ROIs regions of interest
  • U avg_s and U max_s can be calculated by the following equations, where MaxPool and AvgPool represent the max pooling operation and the average pooling operation, respectively.
  • ⁇ represents the Sigmoid activation function
  • Conv represents the convolution operation
  • the convolution kernel size is 3 ⁇ 3. Note that the spatial attention weights will be replicated along the channel axis.
  • the channel attention submodule is used to find internal channel relationships, caring about "what" is of interest in a given feature map.
  • U avg_c and U max_c can be calculated by:
  • PConv point-wise convolution
  • ⁇ represents the ReLU activation function.
  • the convolution kernel size of PConv1 is C/r ⁇ C ⁇ 1 ⁇ 1
  • the convolution kernel size of inverse transform PConv2 is C ⁇ C/r ⁇ 1 ⁇ 1.
  • the ratio r is recommended to be 16, and other scaling ratios can also be selected. It should be noted that the channel attention weights will be replicated along the width and height directions.
  • Our proposed PAM is a plug-and-play module, which ensures strict consistency of input tensors and output tensors at the dimension level; therefore, it can theoretically be embedded in any position of any convolutional neural network model as a supplementary module; this
  • the patent gives two recommended solutions for PAM embedding Hourglass and ResNet.
  • the ResNet network the PAM is embedded in the residual block after the batch normalization layer, before the residual connection, and the same in each residual block. Operation; for the Hourglass network, it is divided into two parts: downsampling and upsampling.
  • the downsampling part embeds the PAM between the residual blocks as a transition module
  • the upsampling part embeds the PAM before the residual connection. Details are shown in Figure 6 and Figure 7, respectively.
  • the unmanned ship system includes four modules, hull module, video acquisition module, lidar navigation module and ground station module.
  • the modules cooperate with each other and work together.
  • the hull module includes the trimaran and power system.
  • the trimaran design can make the ship more stable. It is designed to resist 6-level wind and waves, and the effective remote control distance is 500 meters, which can basically adapt to most engineering application scenarios.
  • the hull size is 75 x 47 x 28 cm for easy transportation.
  • the unmanned ship has an effective load of 5kg and can be equipped with multiple scientific instruments. In addition, the unmanned ship has the function of constant speed cruise, which reduces the control burden of personnel.
  • the video capture module consists of a three-axis camera pan/tilt, a fixed front camera and a light fill.
  • the three-axis camera gimbal supports 10x optical zoom, auto focus, photo taking and 60FPS video recording. This can meet the needs of disease shooting at different scales and different locations.
  • the fixed front camera can easily determine the hull attitude. Through the wireless image transmission device, the picture can be transmitted back to the ground station in real time. On the one hand, it can carry out disease identification, and on the other hand, it can assist in controlling the USV.
  • a controllable LED fill light board which contains 180 high-brightness LED lamp beads.
  • the gimbal carrying the LED fill light board is 3D printed, which can meet the needs of multi-angle fill light.
  • a fixed front-view LED lamp bead is added to provide light source support for the front-view camera.
  • the lidar navigation module includes lidar, minicomputer, a set of transmission system and control system.
  • Lidar can include 2D LiDAR or 3D LiDAR.
  • the effective scanning radius of 2D LiDAR is 12 meters, and the effective scanning radius of 3D LiDAR can be 100 meters.
  • LiDAR can perform 360° omnidirectional scanning. After it is connected to the minicomputer, it can perform real-time mapping of the surrounding environment of the unmanned ship. Through wireless image transmission, the information of the surrounding scene can be transmitted back to the ground station in real time, so as to realize the lidar navigation of the unmanned ship. Based on lidar navigation, unmanned ships no longer need GPS positioning, which is very advantageous in areas with weak GPS signals such as the bottom of bridges and underground culverts.
  • the wireless transmission system supports real-time transmission of 1080P video, and the maximum transmission distance can reach 10 kilometers. Redundant transmission is adopted to ensure link stability and strong anti-interference.
  • the control system consists of wireless image transmission equipment, Pixhawk 2.4.8 flight control, and SKYDROID T12 receiver. Through the flight control and receiver, we can effectively control the equipment on board.
  • the ground station module includes two remote controls and a number of display devices.
  • the main remote control is used to operate the unmanned ship
  • the secondary remote control is used to control the scientific equipment on board
  • the display equipment is used to monitor the real-time information returned by cameras and lidars.
  • the computer is an optional device. On the one hand, it can display the picture in real time, and on the other hand, it can also process the image in real time to identify the disease.
  • the devices cooperate with each other to realize intelligent disease detection without GPS signal.
  • the inventors tested the proposed scheme under the condition of a water system bridge group (eg, Jiulong Lake water system bridge group in Nanjing City, Jiangsu province, China), as shown in FIG. 8 .
  • the 3D LiDAR carried by the unmanned ship is combined with the SLAM algorithm, and the real-time mapping effect is shown in Figure 9.
  • the collected images include three kinds of diseases: cracks, spalling and steel leakage.
  • the pixel resolution of the disease images is 512 ⁇ 512.
  • the Batchsize during training is 2, the Batchsize during testing is 1, and the learning rate is 5 ⁇ 10 -4 .
  • the detection result of the solution proposed by the present invention is shown in FIG. 9 , and the heat map is the visual result directly output by the network, which can provide evidence for the result of target detection.
  • the chosen evaluation metrics are the average precision AP and average recall AR, which are commonly used in the deep learning field. They are the average values of different categories and different images.
  • the calculation process is briefly described below. First introduce a key concept, the intersection of IoU. It is a common concept in the field of target detection. It measures the degree of overlap between the candidate box, that is, the prediction result of the model and the ground-truth bounding box, that is, the ratio of intersection and union, which can be calculated by the following formula.
  • the recall rate can be calculated as
  • the IoU is usually divided into 10 classes, 0.50:0.05:0.95.
  • the AP 50 used in the embodiment is the precision when the IoU threshold is 0.50
  • the AP 75 is the precision when the IoU threshold is 0.75
  • the average precision AP represents the average precision under 10 IoU thresholds, that is,
  • the average recall AR is the maximum recall for each image given 1, 10, and 100 detections. Then averaging over the categories and 10 IoU thresholds, 3 sub-metrics AR 1 , AR 10 and AR 100 can be obtained. Obviously, the closer the values of AP and AR are to 1, the better the test results and the closer to the label.
  • the specific embodiment verifies the effectiveness of the proposed solution and the applicability to complex engineering.
  • the proposed intelligent detection method is more suitable for multi-disease detection with variable slenderness ratio and complex shape.
  • the proposed unmanned ship system also has high robustness and high practicability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Ocean & Marine Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • Fluid Mechanics (AREA)
  • Image Analysis (AREA)

Abstract

一种近水桥梁多类型病害智能检测方法与无人船设备，该方法包括：基础设施病害目标检测网络CenWholeNet和并行注意力模块PAM，其中的CenWholeNet是一种基于深度学习的Anchor-free目标检测网络，包括骨干网络和检测器两部分，用于自动化、高精度检测采集图像中的病害；所述PAM将注意力机制引入神经网络当中，包括空间注意力和通道注意力两部分，用于增强神经网络的表达能力；所述无人船设备包括船体模块、视频采集模块、激光雷达导航模块和地面站模块，支持无需GPS信息的激光雷达导航、视频信息的远距离实时传输和高鲁棒性实时控制，用于自动化采集桥梁底部信息。该方法可以广泛应用于中小桥梁底部等GPS信号微弱且环境复杂的区域的病害检测中。

Description

近水桥梁多类型病害智能检测方法与无人船设备 技术领域
本发明属于土木工程中的结构健康检测领域，具体涉及一种近水桥梁多类型病害智能检测方法与无人船设备。
背景技术
工程结构在服役过程中，由于荷载和环境等影响，会产生诸多病害。这些病害一旦生成，就会极易累积和扩展，从而影响结构使用寿命和整体安全性，甚至影响人民的生命财产安全。近年来，因为缺乏有效检测保养而发生的结构破坏如桥梁垮塌的案例屡见不鲜。因此，对结构进行定期的检测和保养维护至关重要。
传统的基础设施病害检测方法主要是以人工为主，这些方法需要借助繁杂的工具，并且存在效率低下、人力成本高昂、检测盲区大等问题。所以，最近很多学者将智能检测方法和智能检测设备引入了基础设施病害检测领域。智能检测方法以深度学习技术为代表，它已经给很多行业带来了革命性的解决方案，比如医药健康、航空航天和材料科学等，例如公开号为CN111862112A的专利文献，公开了一种基于深度学习的医学图像分割方法，公开号为CN111651916A的专利文献，公开了一种基于深度学习的材料性能预测方法。同样的，使用深度学习技术进行结构病害的智能感知正吸引着越来越多的人的关注。研究人员将深度学习方法应用在不同病害、不同基础设施的检测中。比如混凝土结构裂缝检测、钢筋混凝土结构多病害检测、钢结构锈蚀检测、螺栓松动检测、古建筑病害检测、盾构隧道缺陷检测等等。但是光有智能算法是不够的，要想实现真正的自动检测，还需要智能的检测设备。为了满足不同的检测项目的需要，多种检测机器人被提出和应用。比如桥梁检测无人机，移动式隧道检测车，桥面检测机器人，爬索机器人等等。比如公开号为CN112171692A的专利文献，公开了一种适用于桥梁挠度智能检测的飞行吸附机器人；公开号为CN111413353A的专利文献，公开了一种智能的移动式隧道衬砌病害综合检测设备；公开号为CN111021244A的专利文献，公开了一种正交异性钢桥面板疲劳开裂检测机器人；公开号为CN109978847A的专利文献，公开了一种基于拉索机器人的索套病害识别方法。
这些方法已经解决了很多工程难题，但是我们同样要注意到目前的解决方案的两点突出不足。(1)目前的智能检测方法主要基于Anchor-based方法，即需要预先设定大量的先验框，也就是anchor boxes，故得名Anchor-based方法。比如公开号为CN111062437A的专利文献，公开了一种基于Faster R-CNN模型的桥梁病害目标检测模型，公开号为CN111310558A的专利文献，同样公开了一种基于Faster R-CNN模型的路面病害提取方法，公开号为CN111127399A的专利文献，公开了一种YOLOv3模型的桥墩病害检测方法。Faster R-CNN模型和YOLO系列 模型都是非常经典的Anchor-based方法。Anchor-based方法的第一个突出问题是算法的效果会受到预先设定的先验框的影响。当在处理结构病害这种具有复杂形状、多种长宽比和多种尺寸的特征时，先验框的尺寸与长宽比可能与目标差异过大，这会降低预测结果的召回率。因此，为了提高检测效果往往会预设大量的先验框。这也就带来了Anchor-based方法的第二个突出问题，大量的先验框会引入大量的超参数和设计选择，这会使得模型非常复杂，且计算量较大，计算效率往往不高。所以，传统的智能检测方法并不适合于进行结构病害检测，工程界亟需更加高效简洁、泛化能力更强的新型智能检测算法。(2)目前，智能设备的可检测区域依然十分有限，主要是面向结构外表面等容易检测的区域，例如公开号为CN110894704A的专利文献，公开了一种基于无人机的公路路面表观病害检测方法，公开号为CN111260615A的专利文献，公开了一种基于无人机的桥梁表观病害检测方法。但是，无人机系统对于较为封闭的空间很难奏效，比如大量中小桥梁的底部区域，净空较低，且情况复杂，人工和智能检测设备往往束手无策。以无人机为例，其飞行往往需要有较广阔的无干扰空间、和GPS信号的辅助定位和操控等。但是净空很低的中小桥梁桥底区域GPS信号往往十分微弱，内部情况也十分复杂，无人机飞入会存在信号丢失、碰撞损坏等风险。并且有的区域非常狭小，可能存在有毒气体，人工难以轻易到达。因此，这些区域成为了多年的检测盲区。对这些区域的有效检测也是工程的重点和难点。工程界迫切需要新型的智能检测设备，去检测这种人工和其他智能设备难以检测的区域。
发明内容
为解决上述问题，本发明公开了近水桥梁多类型病害智能检测方法与无人船设备，适合于中小桥梁底部病害自动化、智能化的检测，所提方案包括智能算法和硬件设备两部分。在保证检测精度的同时兼顾检测速度，同时具有较好的泛化能力以及对复杂工程环境的适用性。
为达到上述目的，本发明的技术方案如下：
近水桥梁多类型病害智能检测方法，包括如下组成部分：
组成部分一：智能检测算法：基于深度学习的基础设施病害目标检测网络CenWholeNet；
组成部分二：将并行注意力模块PAM嵌入目标检测网络CenWholeNet中，所述并行注意力模块包括两个子模块，空间注意力子模块和通道注意力子模块；
组成部分三：智能检测设备：基于激光雷达导航的无人船系统，所述无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet包括如下步骤：步骤一：骨干网络：骨干网络用于提取图像的特征；步骤二：检测器：检测器将提取的图像特征转化为计算需要的张量形式，通过损失函数进行优化；步骤三：结果输出：结果输出则 是将张量转化为边界框，实现目标检测的预测结果输出。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤一，骨干网络的具体方法如下：
给定输入图片
Figure PCTCN2021092393-appb-000001
其中W为图像的宽度，H为图像的高度，3表示图片的通道数，即RGB三个通道；通过骨干网络提取输入图像P的特征；本专利推荐采用两种具有影响力的卷积神经网络模型：沙漏网络Hourglass和深度残差网络ResNet，这是两种非常经典的全卷积编码-解码网络；当然，骨干网络的选择并没有明确的限制，其他的特征提取网络都可以作为本模型的骨干网络。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤二，检测器的具体方法如下：
检测器是CenWholeNet的核心，其将骨干网络提取后的特征转换为由4个张量组成的输出集合
Figure PCTCN2021092393-appb-000002
Figure PCTCN2021092393-appb-000003
表示中心关键点热力图，其中C为病害的类别，这里取为C＝3，r为输出步长，也就是下采样比例，默认的步长为4，通过下采样，我们可以极大提高计算效率；设
Figure PCTCN2021092393-appb-000004
Figure PCTCN2021092393-appb-000005
为ground-truth热力图(ground-truth可以理解成标签)，对于类别c来说，位置(i,j)的ground-truth中心点为
Figure PCTCN2021092393-appb-000006
首先计算其下采样的等价位置
Figure PCTCN2021092393-appb-000007
这里
Figure PCTCN2021092393-appb-000008
然后通过一个高斯核函数，将
Figure PCTCN2021092393-appb-000009
映射到张量
Figure PCTCN2021092393-appb-000010
中，Y p可以由下式定义：
Figure PCTCN2021092393-appb-000011
其中，
Figure PCTCN2021092393-appb-000012
和
Figure PCTCN2021092393-appb-000013
表示中心点的位置(x,y)，σ p＝gaussian_radius/3；gaussian_radius表示表示检测框角点偏移的最大半径，该最大半径保证偏移后的检测框与ground-truth检测框的IoU≥t，在所有的实验中都取t＝0.7；将不同中心点对应的所有的Y p整合起来，即得到ground-truth热力图H：
Figure PCTCN2021092393-appb-000014
其中，H c,x,y表示H在位置(c,x,y)处的值，也就是这个位置为中心点的概率；具体得，H c,x,y＝1表征中心关键点，即正样本；显然，H c,x,y＝0为背景，也就是负样本；我们采用focal loss作为一个度量标准来衡量
Figure PCTCN2021092393-appb-000015
和H之间的距离，即
Figure PCTCN2021092393-appb-000016
其中，N为所有中心关键点的数目，α和β是超参数，用来控制权重；在所有的情况下，均取α＝2，β＝4；通过最小化L Heat，神经网络模型可以更好地预测出目标的中心点的位置；
我们需要获取预测框的尺寸信息W×H才能最终确定边界框，设第k个关键点p k对应的ground-truth边界框的尺寸为d k＝(w k,h k)，将所有d k进行整合，即可得到ground-truth边界框尺寸张量
Figure PCTCN2021092393-appb-000017
Figure PCTCN2021092393-appb-000018
其中，
Figure PCTCN2021092393-appb-000019
表示像素级加法；对于所有的病害类别，模型会给出一个预测的维度张量
Figure PCTCN2021092393-appb-000020
我们使用L1 Loss来衡量D和
Figure PCTCN2021092393-appb-000021
的相似性：
Figure PCTCN2021092393-appb-000022
通过最小化L D，模型可以得到每一个预测框的粗略的宽度和高度；
由于图片存在r倍的尺寸缩放(这可以显著提高计算效率，并减少显卡的计算显存消耗)，我们通过引入位置偏移来修正下采样引起的误差；记第k个关键点p k的坐标为(x k,y k)，则映射后的坐标为
Figure PCTCN2021092393-appb-000023
那么可以得到ground-truth偏移：
Figure PCTCN2021092393-appb-000024
整合所有的o k，即可得到ground-truth偏移矩阵：
Figure PCTCN2021092393-appb-000025
这里，第一个维度的2表征关键点(x,y)在W和H两个方向的偏移量；对应的，模型会给出一个预测张量
Figure PCTCN2021092393-appb-000026
我们使用smooth L1 Loss来训练偏移损失：
Figure PCTCN2021092393-appb-000027
此外，为了使模型更加关注目标的整体信息(只有宽度和高度信息是远远不够的)，我们引入了一组新的张量来修正预测框，实验验证这可以显著提高检测精度；具体地，我们将检测框一对角点的连线与x轴的夹角以及检测框的对角线长度作为训练目标；设检测框左上角点和右下角点的坐标为
Figure PCTCN2021092393-appb-000028
和
Figure PCTCN2021092393-appb-000029
所以检测框的对角线长度l k可以计算为：
Figure PCTCN2021092393-appb-000030
两个角点之间的连线的倾角θ k可以由下式计算：
Figure PCTCN2021092393-appb-000031
从而，可以构建一对补充极坐标
Figure PCTCN2021092393-appb-000032
更进一步可以获得ground-truth极坐标矩 阵
Figure PCTCN2021092393-appb-000033
Figure PCTCN2021092393-appb-000034
对应的，模型也会给出一个预测张量
Figure PCTCN2021092393-appb-000035
这里，Polar和
Figure PCTCN2021092393-appb-000036
采用一个L1loss来训练：
Figure PCTCN2021092393-appb-000037
最终，对于每一个位置，模型都会预测出C+6的输出，这会组成集合
Figure PCTCN2021092393-appb-000038
它们也将共享网络的权重；网络总的损失函数可以由下式定义：
L＝L Heat+λ OfL Off+λ DL D+λ PolarL Polar
在所有的实验中，都设λ Off＝10，λ D和λ Polar均取为0.1。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤三，结果输出的具体步骤如下：
结果输出部分要做的工作是从预测的热力图张量
Figure PCTCN2021092393-appb-000039
中提取可能的中心关键点坐标，然后根据对应的
Figure PCTCN2021092393-appb-000040
和
Figure PCTCN2021092393-appb-000041
中的信息得到预测的边界框；显然，
Figure PCTCN2021092393-appb-000042
的数值越大就越有可能是中心点；对于类别c来说，如果点p cxy满足下式，则可以认为p cxy是一个备选的中心点；
Figure PCTCN2021092393-appb-000043
显然，我们并不需要非极大值抑制(NMS)，而是一个3×3的最大池化卷积层，就可以实现备选中心点的提取；设我们选择的中心点集合是
Figure PCTCN2021092393-appb-000044
其中N p为选择的中心点总数；对于其中任一中心点
Figure PCTCN2021092393-appb-000045
我们可以提取对应的尺寸信息
Figure PCTCN2021092393-appb-000046
偏移信息
Figure PCTCN2021092393-appb-000047
和极坐标信息
Figure PCTCN2021092393-appb-000048
首先，根据
Figure PCTCN2021092393-appb-000049
计算出预测框尺寸修正值：
Figure PCTCN2021092393-appb-000050
所以，预测框的具体位置为：
Figure PCTCN2021092393-appb-000051
这里，我们简单将边界框尺寸调整超参数取为α y＝α x＝0.9，β y＝β x＝0.1，并没有涉及复杂的调参过程；值得注意的是，本专利介绍的模型并没有涉及复杂的图像旋转、缩放、翻转等仿射变换技巧，仅仅是一个纯的算法框架；显然，如果超参数数值被进一步优化，图像处理技巧被考虑其中，模型的性能可以进一步提高。
进一步地，组成部分二中所述的并行注意力模块PAM的具体步骤如下：
众所周知，注意力在人类的感知中起着非常重要的作用，人眼或者人耳等器官在获取信息的时候，往往会关注更感兴趣的目标，提高其注意力；而抑制不感兴趣的目标，降低其注意力；从人类的注意力中得到灵感，最近有研究人员提出了一种仿生想法，注意力机制：通过在神经网络中嵌入注意力模块，提高有意义的区域的特征张量的权重，降低没有意义的背景等区域的权重，从而可以提高网络的性能；本专利提出了一种轻量的，即插即用的并行注意力模块PAM，实验验证了PAM可以显著提高神经网络的表达能力；PAM考虑特征图两个维度的注意力，空间注意力和通道注意力，并通过并联的方式进行组合；
给定输入特征图为
Figure PCTCN2021092393-appb-000052
其中，C，H和W分别表示通道、高度和宽度；首先，通过空间注意力子模块实施变换
Figure PCTCN2021092393-appb-000053
然后，通过通道注意力子模块实施变换
Figure PCTCN2021092393-appb-000054
最后得到输出的特征图
Figure PCTCN2021092393-appb-000055
变换主要包括卷积、最大池化操作、均值池化操作和ReLU函数等等；总的计算过程如下：
Figure PCTCN2021092393-appb-000056
其中，
Figure PCTCN2021092393-appb-000057
表示输出像素级张量加法；
空间注意力子模块强调“在哪里”提高注意力，关注感兴趣的区域的位置(ROIs)；首先，沿着通道方向对特征图进行最大池化操作和均值池化操作(这将有效凸显出ROIs)，得到若干个二维图像，
Figure PCTCN2021092393-appb-000058
和
Figure PCTCN2021092393-appb-000059
这里λ 1和λ 2为不同的池化操作权重调整超参数，这里取λ 1＝2，λ 2＝1；U avg_s和U max_s可以由下式计算，MaxPool和AvgPool分别表示最大池化操作和平均池化操作；
Figure PCTCN2021092393-appb-000060
Figure PCTCN2021092393-appb-000061
接着，引入卷积操作，生成空间注意力权重
Figure PCTCN2021092393-appb-000062
空间注意力子模块总的计算流程如下：
Figure PCTCN2021092393-appb-000063
上式可以等价为：
Figure PCTCN2021092393-appb-000064
其中，
Figure PCTCN2021092393-appb-000065
表示像素级张量乘法，σ表示Sigmoid激活函数，Conv表示卷积操作，卷积核尺寸为3×3；需要指出，空间注意力权重将会沿着通道轴复制；
通道注意力子模块用于寻找内部通道的关系，关心给定特征图中“什么”是令人感兴趣的；首先，沿着宽度和高度方向进行均值池化操作和最大池化操作，生成若干个1维向量，
Figure PCTCN2021092393-appb-000066
和
Figure PCTCN2021092393-appb-000067
λ 3和λ 4是不同的池化操作权重调整超参数，这里取λ 3＝2，λ 4＝1；U avg_c和U max_c可以由下式计算：
Figure PCTCN2021092393-appb-000068
Figure PCTCN2021092393-appb-000069
后续，引入point-wise卷积(PConv)作为通道上下文聚合器来实现point-wise的通道间交互；为了降低参数量，PConv被设计成沙漏的形式，设衰减的比例为r；最后，可以得到通道注意力权重
Figure PCTCN2021092393-appb-000070
该子模块的计算流程如下：
Figure PCTCN2021092393-appb-000071
上式也就等价为：
Figure PCTCN2021092393-appb-000072
其中，δ表示ReLU激活函数；PConv1的卷积核尺寸为C/r×C×1×1，逆变换PConv2的卷积核尺寸为C×C/r×1×1；比例r推荐取为16，其他的缩放比例也是可以选择的；需要指出，通道注意力权重将会沿着宽度和高度方向进行复制；
我们提出的PAM是一种即插即用的模块，在尺寸层面，保证了输入张量和输出张量的严格一致；因此理论上可以作为补充模块嵌入任何卷积神经网络模型的任何位置；本专利给出了PAM嵌入Hourglass和ResNet的两种推荐方案，针对ResNet网络，将PAM嵌于残差块中的批标准化层之后，残差连接之前，并在每个残差块中都进行同样的操作；针对Hourglass网络，分为下采样和上采样两个部分，下采样部分，将PAM嵌于残差块之间，作为过渡模块，上采样部分，将PAM嵌于残差连接之前，具体细节见附图。
进一步地，组成部分三中所述的基于激光雷达导航的无人船系统具体细节如下：
无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块，模块之间相互配合，协同工作；
船体模块包括三体船体和动力系统；三体船设计可以使船更加稳定，设计可抵抗6级风浪，有效遥控距离为500米，这基本可以适应绝大部分工程应用场景；船体尺寸为75×47×28厘米，方便运输；无人船有效载重5kg，可以加装多台科学仪器；此外，无人船具备定速巡航的功能，减轻人员的操控负担；
视频采集模块由三轴相机云台、固定前置摄像头和补光器组成；三轴相机云台支持10倍光学变焦、自动对焦、拍照和60FPS的视频录制；这可以满足不同尺度、不同位置的病害拍摄需求；固定前置摄像头可以方便确定船体姿态；通过无线图传设备可以将画面实时传回地 面站，一方面可以进行病害识别，一方面可以辅助控制USV；为了应对中小桥梁桥底等光线不足的工作环境，我们加装了可控的LED补光板，内有180颗高亮度LED灯珠；3D打印了承载LED补光板的云台，这可以满足多角度的补光需求；此外还加装有固定前视的LED灯珠，为前视摄像头提供光源支持；
激光雷达导航模块包括激光雷达、迷你计算机、一套传输系统和控制系统；激光雷达可以进行360°全方位扫描；它与迷你计算机连接后，可以进行无人船周围环境的实时建图；通过无线图传，周围场景的信息可以实时传回地面站，从而实现无人船的激光雷达导航；基于激光雷达导航，无人船不再需要GPS定位，这在桥梁底部、地下暗渠等GPS信号微弱的区域非常有优势；无线传输系统支持1080P视频的实时传输，最大传输距离可达10千米；采用冗余传输，保证了链路稳定，抗干扰性较强；控制系统由无线图传设备、Pixhawk 2.4.8飞控、SKYDROID T12接收机组成；通过飞控和接收机，我们可以对船上设备进行有效的控制；
地面站模块包括两个遥控器和诸多显示设备；主遥控器用来操纵无人船，副遥控器用来控制船载科学设备，显示设备用来监视摄像头和激光雷达实时传回的信息；在实际工程检测中，计算机为可选设备，它一方面可以实时显示画面，一方面也可以对图像进行实时处理，识别病害；设备之间相互配合，实现无需GPS信号的智能病害检测。
本发明的有益效果是：
1.在智能检测算法方面，本发明是Anchor-free目标检测算法在结构病害领域的首次应用。传统的Anchor-based方法的检测结果会受到先验框(也就是anchor boxes)的设定的影响，这也就导致了这种算法处理像结构病害这种具有复杂形状、多种尺寸、多种长宽比的特征时(比如钢筋的长宽比可能很大，剥落的长宽比可能很小)，预设的先验框的尺寸与长细比与目标差异会很大，这会检测结果的召回率偏低。此外，为了达到较好的检测效果，往往要预设大量的先验框。这会引入许多超参数和设计选择。这使得模型的设计更加复杂，同时带来了较大的计算量。与Anchor-based方法相比，本发明提出的方法摒弃了复杂的先验框设定，直接预测关键点和相关向量(即宽度、高度等信息)，将它们组成检测框。本发明的方法更加简单、直接与有效，从根本上解决问题，更加适合于具有复杂特征的工程结构病害的检测。除此以为，本发明考虑了注意力机制对神经网络模型的表达能力的增益效果，提出了一个新颖的、轻量的注意力模块。实验结果显示，本发明提出的方法优于多个具有广泛影响力的神经网络模型，在效率和精度两个维度上达到了综合更优的效果。提出的注意力模块也可以在牺牲可以忽略的计算量的前提下，对不同的神经网络模型起到普遍的增益。
2.在智能检测设备方面，本发明提出了一种不依赖GPS信号的无人船方案用于检测中小桥梁底部病害。由于设计和性能的制约，目前的检测设备进行大量的中小桥梁桥底检测时， 往往束手无策。以无人机为例，其飞行往往需要有较广阔的无干扰空间、需要GPS辅助定位等。但是在净空很低的中小桥梁桥底区域、城市地下暗渠和下水道等，空间比较封闭，GPS信号往往十分微弱，内部情况十分复杂。无人机飞入会存在信号丢失、碰撞损坏等风险。并且有的区域非常狭小，可能存在有毒气体，人工难以轻易到达。因此，工程界迫切需要一种新型的智能检测设备，去检测人工和其他智能设备难以检测的区域。本发明率先提出了适合于较封闭区域病害检测的高鲁棒性无人船系统，实验结果显示，该系统在提高检测效率的同时，可以降低工程人员的安全风险和检测难度，节省大量的人力成本，具有很强工程适用性和广阔的应用前景。此外，本发明提出的系统不仅适合于中小桥梁底部，对城市地下暗渠、下水道等工程场景同样具有较大应用潜力。
附图说明
图1本发明所提整体框架示意图；
图2本发明所提的CenWholeNet网络示意图；
图3本发明所提的注意力模块PAM细节图；
图4本发明所提的无人船系统架构方案图；
图5本发明所提的极坐标补充信息示意图；
图6本发明所提的PAM嵌入ResNet网络方案图；
图7本发明所提的PAM嵌入Hourglass网络方案图；
图8本发明所提的方法在桥梁群的应用示意图；
图9本发明所提的无人船设备实时建图示意图；
图10本发明所提的方法的检测结果示意图；
图11本发明所提的算法框架与其他先进的目标检测算法的检测结果对比表；
图12本发明所提的算法框架与其他先进的目标检测算法训练过程对比。
具体实施方式
下面结合附图和具体实施方式，进一步阐明本发明，应理解下述具体实施方式仅用于说明本发明而不用于限制本发明的范围。在阅读了本发明之后，本领域技术人员对本发明的各种等价形式的修改均落于本申请所附权利要求所限定的范围。
一种近水桥梁多类型病害智能检测方法，该解决方案的整体流程图如图1所示，包括如下组成部分：
组成部分一：智能检测算法：基于深度学习的基础设施病害目标检测网络CenWholeNet，网络细节如图2所示。
组成部分二：将并行注意力模块PAM嵌入目标检测网络CenWholeNet中，所述并行注意 力模块包括两个子模块，空间注意力子模块和通道注意力子模块，具体流程如图3所示。
组成部分三：智能检测设备：基于激光雷达导航的无人船系统，所述无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块。无人船系统架构方案如图4所示。
其中组成部分一中所述的基础设施病害目标检测网络CenWholeNet包括如下步骤：步骤一：骨干网络：骨干网络用于提取图像的特征；步骤二：检测器：检测器将提取的图像特征转化为计算需要的张量形式，通过损失函数进行优化；步骤三：结果输出：结果输出则是将张量转化为边界框，实现目标检测的预测结果输出。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤一，骨干网络的具体方法如下：
给定输入图片
Figure PCTCN2021092393-appb-000073
其中W为图像的宽度，H为图像的高度，3表示图片的通道数，即RGB三个通道。通过骨干网络提取输入图像P的特征。本专利推荐采用两种具有影响力的卷积神经网络模型：沙漏网络Hourglass和深度残差网络ResNet，这是两种非常经典的全卷积编码-解码网络。当然，骨干网络的选择并没有明确的限制，其他的特征提取网络都可以作为本模型的骨干网络。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤二，检测器的具体方法如下：
检测器是CenWholeNet的核心，其将骨干网络提取后的特征转换为由4个张量组成的输出集合
Figure PCTCN2021092393-appb-000074
Figure PCTCN2021092393-appb-000075
表示中心关键点热力图，其中C为病害的类别，这里取为C＝3，r为输出步长，也就是下采样比例，默认的步长为4，通过下采样，我们可以极大提高计算效率。设
Figure PCTCN2021092393-appb-000076
Figure PCTCN2021092393-appb-000077
为ground-truth热力图(ground-truth可以理解成标签)，对于类别c来说，位置(i,j)的ground-truth中心点为
Figure PCTCN2021092393-appb-000078
首先计算其下采样的等价位置
Figure PCTCN2021092393-appb-000079
这里
Figure PCTCN2021092393-appb-000080
然后通过一个高斯核函数，将
Figure PCTCN2021092393-appb-000081
映射到张量
Figure PCTCN2021092393-appb-000082
中。Y p可以由下式定义：
Figure PCTCN2021092393-appb-000083
其中，
Figure PCTCN2021092393-appb-000084
和
Figure PCTCN2021092393-appb-000085
表示中心点的位置(x,y)，σ p＝gaussian_radius/3。gaussian_radius表示表示检测框角点偏移的最大半径，该最大半径保证偏移后的检测框与ground-truth检测框的交并比IoU≥t，在所有的实验中都取t＝0.7。将不同中心点对应的所有的Y p整合起来，即得到ground-truth热力图H：
Figure PCTCN2021092393-appb-000086
其中，H c,x,y表示H在位置(c,x,y)处的值，也就是这个位置为中心点的概率。具体得，H c,x,y＝1表征中心关键点，即正样本；显然，H c,x,y＝0为背景，也就是负样本。我们采用focal loss作为一个度量标准来衡量
Figure PCTCN2021092393-appb-000087
和H之间的距离，即
Figure PCTCN2021092393-appb-000088
其中，N为所有中心关键点的数目，α和β是超参数，用来控制权重。在所有的情况下，均取α＝2，β＝4。通过最小化L Heat，神经网络模型可以更好地预测出目标的中心点的位置。
我们需要获取预测框的尺寸信息W×H才能最终确定边界框，设第k个关键点p k对应的ground-truth边界框的尺寸为d k＝(w k,h k)，将所有d k进行整合，即可得到ground-truth边界框尺寸张量
Figure PCTCN2021092393-appb-000089
Figure PCTCN2021092393-appb-000090
其中，
Figure PCTCN2021092393-appb-000091
表示像素级加法。对于所有的病害类别，模型会给出一个预测的维度张量
Figure PCTCN2021092393-appb-000092
我们使用L1 Loss来衡量D和
Figure PCTCN2021092393-appb-000093
的相似性：
Figure PCTCN2021092393-appb-000094
通过最小化L D，模型可以得到每一个预测框的粗略的宽度和高度。
由于图片存在r倍的尺寸缩放(这可以显著提高计算效率，并减少显卡的计算显存消耗)，我们通过引入位置偏移来修正下采样引起的误差；记第k个关键点p k的坐标为(x k,y k)，则映射后的坐标为
Figure PCTCN2021092393-appb-000095
那么可以得到ground-truth偏移：
Figure PCTCN2021092393-appb-000096
整合所有的o k，即可得到ground-truth偏移矩阵：
Figure PCTCN2021092393-appb-000097
这里，第一个维度的2表征关键点(x,y)在W和H两个方向的偏移量。对应的，模型会给出一个预测张量
Figure PCTCN2021092393-appb-000098
我们使用smooth L1 Loss来训练偏移损失：
Figure PCTCN2021092393-appb-000099
此外，为了使模型更加关注目标的整体信息(只有宽度和高度信息是远远不够的)，我们引入了一组新的张量来修正预测框，实验验证这可以显著提高检测精度。具体地，我们将检测框一对角点的连线与x轴的夹角以及检测框的对角线长度作为训练目标，如图5所示。设 检测框左上角点和右下角点的坐标为
Figure PCTCN2021092393-appb-000100
和
Figure PCTCN2021092393-appb-000101
所以检测框的对角线长度l k可以计算为：
Figure PCTCN2021092393-appb-000102
两个角点之间的连线的倾角θ k可以由下式计算：
Figure PCTCN2021092393-appb-000103
从而，可以构建一对补充极坐标
Figure PCTCN2021092393-appb-000104
更进一步可以获得ground-truth极坐标矩阵
Figure PCTCN2021092393-appb-000105
Figure PCTCN2021092393-appb-000106
对应的，模型也会给出一个预测张量
Figure PCTCN2021092393-appb-000107
这里，Polar和
Figure PCTCN2021092393-appb-000108
采用一个L1loss来训练：
Figure PCTCN2021092393-appb-000109
最终，对于每一个位置，模型都会预测出C+6的输出，这会组成集合
Figure PCTCN2021092393-appb-000110
它们也将共享网络的权重。网络总的损失函数可以由下式定义：
L＝L Heat+λ OfL Off+λ DL D+λ PolarL Polar
在所有的实验中，都设λ Off＝10，λ D和λ Polar均取为0.1。
进一步地，组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤三，结果输出的具体步骤如下：
结果输出部分要做的工作是从预测的热力图张量
Figure PCTCN2021092393-appb-000111
中提取可能的中心关键点坐标，然后根据对应的
Figure PCTCN2021092393-appb-000112
和
Figure PCTCN2021092393-appb-000113
中的信息得到预测的边界框。显然，
Figure PCTCN2021092393-appb-000114
的数值越大就越有可能是中心点。对于类别c来说，如果点p cxy满足下式，则可以认为p cxy是一个备选的中心点。
Figure PCTCN2021092393-appb-000115
显然，我们并不需要非极大值抑制(NMS)，而是一个3×3的最大池化卷积层，就可以实现备选中心点的提取。设我们选择的中心点集合是
Figure PCTCN2021092393-appb-000116
其中N p为选择的中心点总数。对于其中任一中心点
Figure PCTCN2021092393-appb-000117
我们可以提取对应的尺寸信息
Figure PCTCN2021092393-appb-000118
偏移信息
Figure PCTCN2021092393-appb-000119
和极坐标信息
Figure PCTCN2021092393-appb-000120
首先，根据
Figure PCTCN2021092393-appb-000121
计算出预测框尺寸修正值：
Figure PCTCN2021092393-appb-000122
所以，预测框的具体位置为：
Figure PCTCN2021092393-appb-000123
这里，我们简单将边界框尺寸调整超参数取为α y＝α x＝0.9，β y＝β x＝0.1，并没有涉及复杂的调参过程。值得注意的是，本专利介绍的模型并没有涉及复杂的图像旋转、缩放、翻转等仿射变换技巧，仅仅是一个纯的算法框架。显然，如果超参数数值被进一步优化，图像处理技巧被考虑其中，模型的性能可以进一步提高。
进一步地，组成部分二中所述的并行注意力模块PAM的具体步骤如下：
众所周知，注意力在人类的感知中起着非常重要的作用，人眼或者人耳等器官在获取信息的时候，往往会关注更感兴趣的目标，提高其注意力。而抑制不感兴趣的目标，降低其注意力。从人类的注意力中得到灵感，最近有研究人员提出了一种仿生想法，注意力机制：通过在神经网络中嵌入注意力模块，提高有意义的区域的特征张量的权重，降低没有意义的背景等区域的权重，从而可以提高网络的性能。本专利提出了一种轻量的，即插即用的并行注意力模块PAM，实验验证了PAM可以显著提高神经网络的表达能力。PAM考虑特征图两个维度的注意力，空间注意力和通道注意力，并通过并联的方式进行组合。
给定输入特征图为
Figure PCTCN2021092393-appb-000124
其中，C，H和W分别表示通道、高度和宽度。首先，通过空间注意力子模块实施变换
Figure PCTCN2021092393-appb-000125
然后，通过通道注意力子模块实施变换
Figure PCTCN2021092393-appb-000126
最后得到输出的特征图
Figure PCTCN2021092393-appb-000127
变换主要包括卷积、最大池化操作、均值池化操作和ReLU函数等等。总的计算过程如下：
Figure PCTCN2021092393-appb-000128
其中，
Figure PCTCN2021092393-appb-000129
表示输出像素级张量加法。
空间注意力子模块强调“在哪里”提高注意力，关注感兴趣的区域的位置(ROIs)。首先，沿着通道方向对特征图进行最大池化操作和均值池化操作(这将有效凸显出ROIs)，得到若干个二维图像，
Figure PCTCN2021092393-appb-000130
和
Figure PCTCN2021092393-appb-000131
这里λ 1和λ 2为不同的池化操作权重调整超参数，这里取λ 1＝2，λ 2＝1。U avg_s和U max_s可以由下式计算，MaxPool和AvgPool分别表示最大池化操作和平均池化操作。
Figure PCTCN2021092393-appb-000132
Figure PCTCN2021092393-appb-000133
接着，引入卷积操作，生成空间注意力权重
Figure PCTCN2021092393-appb-000134
空间注意力子模块总的计算流程如下：
Figure PCTCN2021092393-appb-000135
上式可以等价为：
Figure PCTCN2021092393-appb-000136
其中，
Figure PCTCN2021092393-appb-000137
表示像素级张量乘法，σ表示Sigmoid激活函数，Conv表示卷积操作，卷积核尺寸为3×3。需要指出，空间注意力权重将会沿着通道轴复制。
通道注意力子模块用于寻找内部通道的关系，关心给定特征图中“什么”是令人感兴趣的。首先，沿着宽度和高度方向进行均值池化操作和最大池化操作，生成若干个1维向量，
Figure PCTCN2021092393-appb-000138
和
Figure PCTCN2021092393-appb-000139
λ 3和λ 4是不同的池化操作权重调整超参数，这里取λ 3＝2，λ 4＝1。U avg_c和U max_c可以由下式计算：
Figure PCTCN2021092393-appb-000140
Figure PCTCN2021092393-appb-000141
后续，引入point-wise卷积(PConv)作为通道上下文聚合器来实现point-wise的通道间交互。为了降低参数量，PConv被设计成沙漏的形式，设衰减的比例为r。最后，可以得到通道注意力权重
Figure PCTCN2021092393-appb-000142
该子模块的计算流程如下：
Figure PCTCN2021092393-appb-000143
上式也就等价为：
Figure PCTCN2021092393-appb-000144
其中，δ表示ReLU激活函数。PConv1的卷积核尺寸为C/r×C×1×1，逆变换PConv2的卷积核尺寸为C×C/r×1×1。比例r推荐取为16，其他的缩放比例也是可以选择的。需要指出，通道注意力权重将会沿着宽度和高度方向进行复制。
我们提出的PAM是一种即插即用的模块，在尺寸层面，保证了输入张量和输出张量的严格一致；因此理论上可以作为补充模块嵌入任何卷积神经网络模型的任何位置；本专利给出了PAM嵌入Hourglass和ResNet的两种推荐方案，针对ResNet网络，将PAM嵌于残差块中的批标准化层之后，残差连接之前，并在每个残差块中都进行同样的操作；针对Hourglass网络，分为下采样和上采样两个部分，下采样部分，将PAM嵌于残差块之间，作为过渡模块，上采样部分，将PAM嵌于残差连接之前，具体嵌入细节分别见图6和图7。
进一步地，组成部分三中所述的基于激光雷达导航的无人船系统具体细节如下：
无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块，模块之间相互配合，协同工作。
船体模块包括三体船体和动力系统。三体船设计可以使船更加稳定，设计可抵抗6级风 浪，有效遥控距离为500米，这基本可以适应绝大部分工程应用场景。船体尺寸为75×47×28厘米，方便运输。无人船有效载重5kg，可以加装多台科学仪器。此外，无人船具备定速巡航的功能，减轻人员的操控负担。
视频采集模块由三轴相机云台、固定前置摄像头和补光器组成。三轴相机云台支持10倍光学变焦、自动对焦、拍照和60FPS的视频录制。这可以满足不同尺度、不同位置的病害拍摄需求。固定前置摄像头可以方便确定船体姿态。通过无线图传设备可以将画面实时传回地面站，一方面可以进行病害识别，一方面可以辅助控制USV。为了应对中小桥梁桥底等光线不足的工作环境，我们加装了可控的LED补光板，内有180颗高亮度LED灯珠。3D打印了承载LED补光板的云台，这可以满足多角度的补光需求。此外还加装有固定前视的LED灯珠，为前视摄像头提供光源支持。
激光雷达导航模块包括激光雷达、迷你计算机、一套传输系统和控制系统。激光雷达可以包括二维激光雷达，或三维激光雷达，二维激光雷达的有效扫描半径12米，三维激光雷达的有效扫描半径可以是100米，激光雷达可以进行360°全方位扫描。它与迷你计算机连接后，可以进行无人船周围环境的实时建图。通过无线图传，周围场景的信息可以实时传回地面站，从而实现无人船的激光雷达导航。基于激光雷达导航，无人船不再需要GPS定位，这在桥梁底部、地下暗渠等GPS信号微弱的区域非常有优势。无线传输系统支持1080P视频的实时传输，最大传输距离可达10千米。采用冗余传输，保证了链路稳定，抗干扰性较强。控制系统由无线图传设备、Pixhawk 2.4.8飞控、SKYDROID T12接收机组成。通过飞控和接收机，我们可以对船上设备进行有效的控制。
地面站模块包括两个遥控器和诸多显示设备。主遥控器用来操纵无人船，副遥控器用来控制船载科学设备，显示设备用来监视摄像头和激光雷达实时传回的信息。在实际工程检测中，计算机为可选设备，它一方面可以实时显示画面，一方面也可以对图像进行实时处理，识别病害。设备之间相互配合，实现无需GPS信号的智能病害检测。
实施例1
发明人在水系桥梁群(例如，中国江苏省南京市的九龙湖水系桥梁群)的条件下，对所提方案进行了检验，如图8所示。无人船搭载的三维激光雷达结合SLAM算法，实时建图效果如9所示。该桥梁群内共有5座中小桥梁，采集的图像包括三种病害：裂缝、剥落和钢筋漏出，病害图像的像素分辨率为512×512。基于PyTorch深度学习框架进行模型的搭建、训练和测试。训练时的Batchsize取为2，测试时的Batchsize取为1，学习率取为5×10 -4。本发明所提方案的检测结果如图9所示，热力图为网络直接输出的可视化结果，可以对目标检测的结果提供佐证。
我们还在相同数据集上，将本发明所提方法与最先进的目标检测模型进行了对比，包括Anchor-based方法中具有广泛影响力的目标检测方法Faster R-CNN方法和在工业界中得到广泛应用的YOLO方法中最新的YOLOv5模型，Anchor-free中广受好评的CenterNet方法。此外，我们还将我们提出的注意力模块PAM与深度学习社区公认的优秀的、经典的注意力模块SENet和CBAM进行了对比。
选择的评估指标是深度学习领域中普遍采用的平均精度AP和平均召回率AR。它们均是不同类别、不同图像下的平均值，下面简要叙述计算过程。首先介绍一个关键概念，交并比IoU。它是目标检测领域中的常用概念，衡量候选框也就是模型的预测结果和ground-truth边界框的重叠程度，即交集与并集的比值，可以由以下公式计算。
Figure PCTCN2021092393-appb-000145
对于每一个预测框，其与ground-truth边界框之间考虑3种关系。与ground-truth边界框的IoU大于规定的阈值的预测框数量，则记为真正类TP；与ground truth边界框的IoU小于阈值的预测框数量，记为假正类FP，未检测到的ground truth边界框的数量，记为假负类FP。则准确率可以计算为
Figure PCTCN2021092393-appb-000146
召回率可以计算为
Figure PCTCN2021092393-appb-000147
因此，根据IoU阈值的不同，可以计算出不同的精度。通常将IoU划分为10类，0.50:0.05:0.95。实施例中用到的AP 50是IoU阈值为0.50时的精度，AP 75是IoU阈值为0.75时的精度，平均精度AP表示10个IoU阈值下的平均精度，即
Figure PCTCN2021092393-appb-000148
这是衡量模型检测性能最重要的指标。平均召回率AR是每张图片上，给定1、10和100次检测下，产生的最大召回。然后在类别和10个IoU阈值下进行平均，可以得到3个子指标AR 1，AR 10和AR 100。显然，AP和AR的数值越接近1，则测试结果越好，越贴近标签。
不同方法间的预测结果对比如下图10所示，其中参数量是一个恒量深度学习模型“体积”的量。FPS(frame-per-second)表示算法1秒钟处理图像的数目，也就是表征了算法的运行速度。与Faster R-CNN方法相比，本发明提出的方法在效率和精度两个维度上，都明显优于Faster R-CNN。与YOLO v5的4个子版本YOLO v5s，YOLO v5m，YOLO v5l和YOLO v5x都进行了对比，可以看到效果并不是十分理想，我们对YOLOv5的差检测结果感到非常震惊。只能将最好的YOLO v5子版本YOLO v5x训练了更多的Epoch，才获得了可以比较的性能。虽然运行速度上，YOLO v5稍快，但是精度上远远不如本文所提方法。与CenterNet方法相比，运 行速度相同，但是检测效果远高于CenterNet。在注意力模块层面上的比较可以得出两个结论：(1)本发明提出的PAM可以在牺牲少量计算量的前提下，对不同的深度学习模型起到普遍的、大幅的增益效果；(2)和SENet和CBAM相比，PAM可以获得更多的增益加成，明显优于SENet和CBAM。
不同方法间的训练过程对比如图11所示，本发明所提方法为圆圈标注线。可以很明显看到，虽然训练的结果会发生不同程度的震荡，但是与传统方法相比，我们的方法总体上均可以获得更高的AP和AR。即可以获得更好的目标检测效果。
综上，具体实施例验证了本发明所提方案的有效性和对复杂工程的适用性。与传统深度学习方法相比，所提智能检测方法更加适合于长细比多变和形态复杂的多病害检测。所提无人船系统也具有高鲁棒性和高实用性。
以上公开的仅为本发明的一个典型实施例，但是，本发明实施例并非局限于此，任何本领域的技术人员经过阅读专利之后，对本专利进行的同质修改都应落入本发明的保护范围。

Claims (7)

  1. 近水桥梁多类型病害智能检测方法，其特征在于：包括如下组成部分：
    组成部分一：智能检测算法：基于深度学习的基础设施病害目标检测网络CenWholeNet；
    组成部分二：将并行注意力模块PAM嵌入目标检测网络CenWholeNet中，所述并行注意力模块包括两个子模块：空间注意力子模块和通道注意力子模块；
    组成部分三：智能检测设备：基于激光雷达导航的无人船系统，所述无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块。
  2. 根据权利要求1所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分一中所述的基础设施病害目标检测网络CenWholeNet包括如下步骤：步骤一：骨干网络：骨干网络用于提取图像的特征；步骤二：检测器：检测器将提取的图像特征转化为计算需要的张量形式，通过损失函数进行优化；步骤三：结果输出：结果输出则是将张量转化为边界框，实现目标检测的预测结果输出。
  3. 根据权利要求2所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤一，骨干网络的具体方法如下：
    给定输入图片
    Figure PCTCN2021092393-appb-100001
    其中W为图像的宽度，H为图像的高度，3表示图片的通道数，即RGB三个通道；通过骨干网络提取输入图像P的特征；采用两种卷积神经网络模型：沙漏网络Hourglass和深度残差网络ResNet。
  4. 根据权利要求2所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤二，检测器的具体方法如下：
    检测器是CenWholeNet的核心，其将骨干网络提取后的特征转换为由4个张量组成的输出集合
    Figure PCTCN2021092393-appb-100002
    Figure PCTCN2021092393-appb-100003
    表示中心关键点热力图，其中C为病害的类别，这里取为C＝3，r为输出步长，也就是下采样比例，默认的步长为4，通过下采样，提高计算效率；设
    Figure PCTCN2021092393-appb-100004
    为ground-truth热力图，对于类别c来说，位置(i,j)的ground-truth中心点为
    Figure PCTCN2021092393-appb-100005
    首先计算其下采样的等价位置
    Figure PCTCN2021092393-appb-100006
    这里
    Figure PCTCN2021092393-appb-100007
    然后通过一个高斯核函数，将
    Figure PCTCN2021092393-appb-100008
    映射到张量
    Figure PCTCN2021092393-appb-100009
    中，Y p下式定义：
    Figure PCTCN2021092393-appb-100010
    其中，
    Figure PCTCN2021092393-appb-100011
    和
    Figure PCTCN2021092393-appb-100012
    表示中心点的位置(x,y)，σ p＝gaussian_radius/3；gaussian_radius表示表示检测框角点偏移的最大半径，该最大半径保证偏移后的检测框与ground-truth检测框的交 并比IoU≥t，在所有的实验中都取t＝0.7；将不同中心点对应的所有的Y p整合起来，即得到ground-truth热力图H：
    Figure PCTCN2021092393-appb-100013
    其中，H c,x,y表示H在位置(c,x,y)处的值，也就是这个位置为中心点的概率；具体得，H c,x,y＝1表征中心关键点，即正样本；显然，H c,x,y＝0为背景，也就是负样本；我们采用focal loss作为一个度量标准来衡量
    Figure PCTCN2021092393-appb-100014
    和H之间的距离，即
    Figure PCTCN2021092393-appb-100015
    其中，N为所有中心关键点的数目，α和β是超参数，用来控制权重；在所有的情况下，均取α＝2，β＝4；通过最小化L Heat，神经网络模型更好地预测出目标的中心点的位置；
    我们需要获取预测框的尺寸信息W×H才能最终确定边界框，设第k个关键点p k对应的ground-truth边界框的尺寸为d k＝(w k,h k)，将所有d k进行整合，得到ground-truth边界框尺寸张量
    Figure PCTCN2021092393-appb-100016
    Figure PCTCN2021092393-appb-100017
    其中，
    Figure PCTCN2021092393-appb-100018
    表示像素级加法；对于所有的病害类别，模型会给出一个预测的维度张量
    Figure PCTCN2021092393-appb-100019
    我们使用L1 Loss来衡量D和
    Figure PCTCN2021092393-appb-100020
    的相似性：
    Figure PCTCN2021092393-appb-100021
    通过最小化L D，模型得到每一个预测框的粗略的宽度和高度；
    由于图片存在r倍的尺寸缩放，我们通过引入位置偏移来修正下采样引起的误差；记第k个关键点p k的坐标为(x k,y k)，则映射后的坐标为
    Figure PCTCN2021092393-appb-100022
    那么得到ground-truth偏移：
    Figure PCTCN2021092393-appb-100023
    整合所有的o k，得到ground-truth偏移矩阵
    Figure PCTCN2021092393-appb-100024
    Figure PCTCN2021092393-appb-100025
    这里，第一个维度的2表征关键点(x,y)在W和H两个方向的偏移量；对应的，模型会给出一个预测张量
    Figure PCTCN2021092393-appb-100026
    我们使用smooth L1 Loss来训练偏移损失：
    Figure PCTCN2021092393-appb-100027
    此外，为了使模型更加关注目标的整体信息，我们引入了一组新的张量来修正预测框，提高检测精度；具体地，我们将检测框一对角点的连线与x轴的夹角以及检测框的对角线长度作为训练目标；设检测框左上角点和右下角点的坐标为
    Figure PCTCN2021092393-appb-100028
    和
    Figure PCTCN2021092393-appb-100029
    所以检测框的对角线长度l k计算为：
    Figure PCTCN2021092393-appb-100030
    两个角点之间的连线的倾角θ k由下式计算：
    Figure PCTCN2021092393-appb-100031
    从而构建一对补充极坐标
    Figure PCTCN2021092393-appb-100032
    更进一步获得ground-truth极坐标矩阵
    Figure PCTCN2021092393-appb-100033
    Figure PCTCN2021092393-appb-100034
    Figure PCTCN2021092393-appb-100035
    对应的，模型也会给出一个预测张量
    Figure PCTCN2021092393-appb-100036
    这里，Polar和
    Figure PCTCN2021092393-appb-100037
    采用一个L1 loss来训练：
    Figure PCTCN2021092393-appb-100038
    最终，对于每一个位置，模型都会预测出C+6的输出，这会组成集合
    Figure PCTCN2021092393-appb-100039
    它们也将共享网络的权重；网络总的损失函数由下式定义：
    L＝L Heat+λ OfL Off+λ DL D+λ PolarL Polar
    在所有的实验中，都设λ Off＝10，λ D和λ Polar均取为0.1。
  5. 根据权利要求2所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分一中所述的基础设施病害目标检测网络CenWholeNet的步骤三，结果输出的具体步骤如下：
    结果输出部分要做的工作是从预测的热力图张量
    Figure PCTCN2021092393-appb-100040
    中提取可能的中心关键点坐标，然后根据对应的
    Figure PCTCN2021092393-appb-100041
    和
    Figure PCTCN2021092393-appb-100042
    中的信息得到预测的边界框；显然，
    Figure PCTCN2021092393-appb-100043
    的数值越大就越有可能是中心点；对于类别c来说，如果点p cxy满足下式，则认为p cxy是一个备选的中心点；
    Figure PCTCN2021092393-appb-100044
    显然，我们并不需要非极大值抑制(NMS)，而是一个3×3的最大池化卷积层，实现备选中心 点的提取；设我们选择的中心点集合是
    Figure PCTCN2021092393-appb-100045
    其中N p为选择的中心点总数；对于其中任一中心点
    Figure PCTCN2021092393-appb-100046
    提取对应的尺寸信息
    Figure PCTCN2021092393-appb-100047
    偏移信息
    Figure PCTCN2021092393-appb-100048
    Figure PCTCN2021092393-appb-100049
    和极坐标信息
    Figure PCTCN2021092393-appb-100050
    首先，根据
    Figure PCTCN2021092393-appb-100051
    计算出预测框尺寸修正值：
    Figure PCTCN2021092393-appb-100052
    所以，预测框的具体位置为：
    Figure PCTCN2021092393-appb-100053
    这里，我们将边界框尺寸调整超参数取为α y＝α x＝0.9，β y＝β x＝0.1。
  6. 根据权利要求1所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分二中所述的并行注意力模块PAM的具体步骤如下：
    本专利提出了一种轻量的，即插即用的并行注意力模块PAM，实验验证了PAM显著提高神经网络的表达能力；PAM考虑特征图两个维度的注意力，空间注意力和通道注意力，并通过并联的方式进行组合；
    给定输入特征图为
    Figure PCTCN2021092393-appb-100054
    其中，C，H和W分别表示通道、高度和宽度；首先，通过空间注意力子模块实施变换
    Figure PCTCN2021092393-appb-100055
    然后，通过通道注意力子模块实施变换
    Figure PCTCN2021092393-appb-100056
    最后得到输出的特征图
    Figure PCTCN2021092393-appb-100057
    变换主要包括卷积、最大池化操作、均值池化操作和ReLU函数；总的计算过程如下：
    Figure PCTCN2021092393-appb-100058
    其中，
    Figure PCTCN2021092393-appb-100059
    表示输出像素级张量加法；
    空间注意力子模块强调“在哪里”提高注意力，关注感兴趣的区域的位置(ROIs)；首先，沿着通道方向对特征图进行最大池化操作和均值池化操作，得到若干个二维图像，
    Figure PCTCN2021092393-appb-100060
    Figure PCTCN2021092393-appb-100061
    和
    Figure PCTCN2021092393-appb-100062
    这里λ 1和λ 2为不同的池化操作权重调整超参数，这里取λ 1＝2，λ 2＝1；U avg_s和U max_s由下式计算，MaxPool和AvgPool分别表示最大池化操作和平均池化操作；
    Figure PCTCN2021092393-appb-100063
    Figure PCTCN2021092393-appb-100064
    接着，引入卷积操作，生成空间注意力权重
    Figure PCTCN2021092393-appb-100065
    空间注意力子模块总的计算流程如下：
    Figure PCTCN2021092393-appb-100066
    上式等价为：
    Figure PCTCN2021092393-appb-100067
    其中，
    Figure PCTCN2021092393-appb-100068
    表示像素级张量乘法，σ表示Sigmoid激活函数，Conv表示卷积操作，卷积核尺寸为3×3；需要指出，空间注意力权重将会沿着通道轴复制；
    通道注意力子模块用于寻找内部通道的关系，关心给定特征图中“什么”是令人感兴趣的；首先，沿着宽度和高度方向进行均值池化操作和最大池化操作，生成若干个1维向量，
    Figure PCTCN2021092393-appb-100069
    和
    Figure PCTCN2021092393-appb-100070
    λ 3和λ 4是不同的池化操作权重调整超参数，这里取λ 3＝2，λ 4＝1；U avg_c和U max_c由下式计算：
    Figure PCTCN2021092393-appb-100071
    Figure PCTCN2021092393-appb-100072
    后续，引入point-wise卷积(PConv)作为通道上下文聚合器来实现point-wise的通道间交互；为了降低参数量，PConv被设计成沙漏的形式，设衰减的比例为r；最后得到通道注意力权重
    Figure PCTCN2021092393-appb-100073
    该子模块的计算流程如下：
    Figure PCTCN2021092393-appb-100074
    上式也就等价为：
    Figure PCTCN2021092393-appb-100075
    其中，δ表示ReLU激活函数；PConv1的卷积核尺寸为C/r×C×1×1，逆变换PConv2的卷积核尺寸为C×C/r×1×1；比例r推荐取为16，需要指出，通道注意力权重将会沿着宽度和高度方向进行复制；
    我们提出的PAM是一种即插即用的模块，在尺寸层面，保证了输入张量和输出张量的严格一致；因此理论上PAM作为补充模块嵌入任何卷积神经网络模型的任何位置；本专利给出了PAM嵌入Hourglass和ResNet的两种推荐方案，针对ResNet网络，将PAM嵌于残差块中的批标准化层之后，残差连接之前，并在每个残差块中都进行同样的操作；针对Hourglass网络，分为下采样和上采样两个部分，下采样部分，将PAM嵌于残差块之间，作为过渡模块，上采样部分，将PAM嵌于残差连接之前。
  7. 根据权利要求1所述的近水桥梁多类型病害智能检测方法，其特征在于：组成部分三中所述的基于激光雷达导航的无人船系统具体细节如下：
    无人船系统包括四个模块，船体模块、视频采集模块、激光雷达导航模块和地面站模块，模块之间相互配合，协同工作；
    船体模块包括三体船体和动力系统；三体船更加稳定，能够抵抗6级风浪，有效遥控距离为500米，能够适应工程应用场景；船体尺寸为75×47×28厘米，方便运输；无人船有效载重5kg，加装多台科学仪器；此外，无人船具备定速巡航的功能，减轻人员的操控负担；
    视频采集模块由三轴相机云台、固定前置摄像头和补光器组成；三轴相机云台支持10倍光学变焦、自动对焦、拍照和60FPS的视频录制；满足不同尺度、不同位置的病害拍摄需求；固定前置摄像头能够方便确定船体姿态；通过无线图传设备将画面实时传回地面站，一方面进行病害识别，一方面辅助控制USV；为了应对中小桥梁桥底等光线不足的工作环境，加装可控的LED补光板，内有180颗高亮度LED灯珠；3D打印了承载LED补光板的云台，满足多角度的补光需求；此外还加装有固定前视的LED灯珠，为前视摄像头提供光源支持；
    激光雷达导航模块包括激光雷达、迷你计算机、一套传输系统和控制系统；激光雷达能够进行360°全方位扫描；它与迷你计算机连接后，能够进行无人船周围环境的实时建图；通过无线图传，周围场景的信息实时传回地面站，从而实现无人船的激光雷达导航；基于激光雷达导航，无人船不再需要GPS定位，这在桥梁底部、地下暗渠这些GPS信号微弱的区域非常有优势；无线传输系统支持1080P视频的实时传输，最大传输距离达10千米；采用冗余传输，保证了链路稳定，抗干扰性较强；控制系统由无线图传设备、Pixhawk 2.4.8飞控、SKYDROID T12接收机组成；通过飞控和接收机，对船上设备进行有效的控制；
    地面站模块包括两个遥控器和诸多显示设备；主遥控器用来操纵无人船，副遥控器用来控制船载科学设备，显示设备用来监视摄像头和激光雷达实时传回的信息；在实际工程检测中，计算机为可选设备，它一方面实时显示画面，一方面对图像进行实时处理，识别病害；设备之间相互配合，实现无需GPS信号的智能病害检测。
PCT/CN2021/092393 2021-03-17 2021-05-08 近水桥梁多类型病害智能检测方法与无人船设备 WO2022193420A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/755,086 US20230351573A1 (en) 2021-03-17 2021-05-08 Intelligent detection method and unmanned surface vehicle for multiple type faults of near-water bridges

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110285996.5A CN112884760B (zh) 2021-03-17 2021-03-17 近水桥梁多类型病害智能检测方法与无人船设备
CN202110285996.5 2021-03-17

Publications (1)

Publication Number Publication Date
WO2022193420A1 true WO2022193420A1 (zh) 2022-09-22

Family

ID=76041072

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/092393 WO2022193420A1 (zh) 2021-03-17 2021-05-08 近水桥梁多类型病害智能检测方法与无人船设备

Country Status (3)

Country Link
US (1) US20230351573A1 (zh)
CN (1) CN112884760B (zh)
WO (1) WO2022193420A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115574785A (zh) * 2022-12-12 2023-01-06 河海大学 基于数据处理的水利工程安全监测方法及平台
CN115953408A (zh) * 2023-03-15 2023-04-11 国网江西省电力有限公司电力科学研究院 一种基于YOLOv7的避雷器表面缺陷检测方法
CN116295020A (zh) * 2023-05-22 2023-06-23 山东高速工程检测有限公司 一种桥梁病害定位方法及定位装置
CN117054891A (zh) * 2023-10-11 2023-11-14 中煤科工(上海)新能源有限公司 电池寿命的预测方法、预测装置
CN117272245A (zh) * 2023-11-21 2023-12-22 陕西金元新能源有限公司 一种风机齿轮箱温度预测方法、装置、设备和介质
CN117710755A (zh) * 2024-02-04 2024-03-15 江苏未来网络集团有限公司 一种基于深度学习的车辆属性识别系统及方法
CN118014997A (zh) * 2024-04-08 2024-05-10 湖南联智智能科技有限公司 一种基于改进yolov5的路面病害识别方法
CN118034269A (zh) * 2024-01-12 2024-05-14 淮阴工学院 一种用于船舶智能机动的自适应控制方法
CN118298340A (zh) * 2024-06-06 2024-07-05 北京理工大学长三角研究院(嘉兴) 一种基于先验知识的无人机航拍密集目标检测方法

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113256601B (zh) * 2021-06-10 2022-09-13 北方民族大学 路面病害检测方法及系统
CN113627245B (zh) * 2021-07-02 2024-01-19 武汉纺织大学 Crts目标检测方法
CN113989614B (zh) * 2021-10-19 2024-10-15 南京航空航天大学 一种路面损伤目标检测的后处理和对角线评价方法
CN113870265B (zh) * 2021-12-03 2022-02-22 绵阳职业技术学院 一种工业零件表面缺陷检测方法
CN114266299A (zh) * 2021-12-16 2022-04-01 京沪高速铁路股份有限公司 基于无人机作业的铁路桥梁钢结构缺陷检测方法及系统
CN114266892A (zh) * 2021-12-20 2022-04-01 江苏燕宁工程科技集团有限公司 一种多源数据深度学习的路面病害识别方法及系统
CN114663774B (zh) * 2022-05-24 2022-12-02 之江实验室 一种轻量级的显著性物体检测系统及方法
CN114820620B (zh) * 2022-06-29 2022-09-13 中冶建筑研究总院（深圳）有限公司 一种螺栓松动缺陷检测方法、系统及装置
CN115061113B (zh) * 2022-08-19 2022-11-01 南京隼眼电子科技有限公司 用于雷达的目标检测模型训练方法、装置及存储介质
CN116740704B (zh) * 2023-06-16 2024-02-27 安徽农业大学 基于深度学习的小麦叶片表型参数变化率监测方法和装置
CN116777895B (zh) * 2023-07-05 2024-05-31 重庆大学 基于可解释深度学习的混凝土桥梁表观病害智能检测方法
CN117333845A (zh) * 2023-11-03 2024-01-02 东北电力大学 一种基于改进YOLOv5s的小目标交通标志实时检测方法
CN117218329B (zh) * 2023-11-09 2024-01-26 四川泓宝润业工程技术有限公司 一种井口阀门检测方法、装置、存储介质及电子设备
CN117541922B (zh) * 2023-11-09 2024-08-06 国网宁夏电力有限公司建设分公司 基于SF-YOLOv5的电站屋面工程缺陷检测方法
CN117724137B (zh) * 2023-11-21 2024-08-06 江苏北斗星通汽车电子有限公司 一种基于多模态传感器的汽车事故自动检测系统及方法
CN117705800A (zh) * 2023-11-23 2024-03-15 华南理工大学 一种基于导轨滑动的机械臂视觉桥梁检测系统及其控制方法
CN117390407B (zh) * 2023-12-13 2024-04-05 国网山东省电力公司济南供电公司 变电站设备的故障识别方法、系统、介质和设备
CN117456610B (zh) * 2023-12-21 2024-04-12 浪潮软件科技有限公司 一种攀爬异常行为检测方法、系统及电子设备
CN117689731B (zh) * 2024-02-02 2024-04-26 陕西德创数字工业智能科技有限公司 一种基于改进YOLOv5模型的轻量化新能源重卡电池包识别方法
CN117727104B (zh) * 2024-02-18 2024-05-07 厦门瑞为信息技术有限公司 基于双边注意力的近红外活体检测装置与方法
CN117788471B (zh) * 2024-02-27 2024-04-26 南京航空航天大学 一种基于YOLOv5检测飞机蒙皮缺陷和分类的方法
CN117805658B (zh) * 2024-02-29 2024-05-10 东北大学 一种数据驱动的电动汽车电池剩余寿命预测方法
CN118050729B (zh) * 2024-04-15 2024-07-09 南京信息工程大学 一种基于改进U-Net的雷达回波时间降尺度订正方法
CN118468204A (zh) * 2024-05-24 2024-08-09 杭州电子科技大学 一种基于改进深度学习模型的制冷机组故障检测方法
CN118396071B (zh) * 2024-07-01 2024-09-03 山东科技大学 一种用于无人船环境理解的边界驱动神经网络结构
CN118604006A (zh) * 2024-08-09 2024-09-06 佛山大学 一种建筑墙体安全检测方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107839845A (zh) * 2017-11-14 2018-03-27 江苏领安智能桥梁防护有限公司 一种无人监测艇
CN108288269A (zh) * 2018-01-24 2018-07-17 东南大学 基于无人机与卷积神经网络的桥梁支座病害自动识别方法
CN109300126A (zh) * 2018-09-21 2019-02-01 重庆建工集团股份有限公司 一种基于空间位置的桥梁病害高精度智能检测方法
CN112465748A (zh) * 2020-11-10 2021-03-09 西南科技大学 基于神经网络的裂缝识别方法、装置、设备及存储介质
CN112488990A (zh) * 2020-11-02 2021-03-12 东南大学 一种基于注意力正则化机制的桥梁支座病害识别方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10489976B2 (en) * 2017-08-11 2019-11-26 Jing Jin Incident site investigation and management support system based on unmanned aerial vehicles
CA3099443A1 (en) * 2017-11-02 2019-05-09 Airworks Solutions, Inc. Methods and apparatus for automatically defining computer-aided design files using machine learning, image analytics, and/or computer vision
US10354169B1 (en) * 2017-12-22 2019-07-16 Motorola Solutions, Inc. Method, device, and system for adaptive training of machine learning models via detected in-field contextual sensor events and associated located and retrieved digital audio and/or video imaging
US11170299B2 (en) * 2018-12-28 2021-11-09 Nvidia Corporation Distance estimation to objects and free-space boundaries in autonomous machine applications
CN110222701B (zh) * 2019-06-11 2019-12-27 北京新桥技术发展有限公司 一种桥梁病害自动识别方法
CN111324126B (zh) * 2020-03-12 2022-07-05 集美大学 一种视觉无人船
CN111507271B (zh) * 2020-04-20 2021-01-12 北京理工大学 一种机载光电视频目标智能化检测与识别方法
CN112069868A (zh) * 2020-06-28 2020-12-11 南京信息工程大学 一种基于卷积神经网络的无人机实时车辆检测方法
CN112329827B (zh) * 2020-10-26 2022-08-23 同济大学 一种基于元学习的增量小样本目标检测方法
US11521357B1 (en) * 2020-11-03 2022-12-06 Bentley Systems, Incorporated Aerial cable detection and 3D modeling from images
US20240020953A1 (en) * 2022-07-15 2024-01-18 Nvidia Corporation Surround scene perception using multiple sensors for autonomous systems and applications

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107839845A (zh) * 2017-11-14 2018-03-27 江苏领安智能桥梁防护有限公司 一种无人监测艇
CN108288269A (zh) * 2018-01-24 2018-07-17 东南大学 基于无人机与卷积神经网络的桥梁支座病害自动识别方法
CN109300126A (zh) * 2018-09-21 2019-02-01 重庆建工集团股份有限公司 一种基于空间位置的桥梁病害高精度智能检测方法
CN112488990A (zh) * 2020-11-02 2021-03-12 东南大学 一种基于注意力正则化机制的桥梁支座病害识别方法
CN112465748A (zh) * 2020-11-10 2021-03-09 西南科技大学 基于神经网络的裂缝识别方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JUN FU; JING LIU; HAIJIE TIAN; YONG LI; YONGJUN BAO; ZHIWEI FANG; HANQING LU: "Dual Attention Network for Scene Segmentation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 9 September 2018 (2018-09-09), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081201270 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115574785A (zh) * 2022-12-12 2023-01-06 河海大学 基于数据处理的水利工程安全监测方法及平台
CN115574785B (zh) * 2022-12-12 2023-02-28 河海大学 基于数据处理的水利工程安全监测方法及平台
CN115953408A (zh) * 2023-03-15 2023-04-11 国网江西省电力有限公司电力科学研究院 一种基于YOLOv7的避雷器表面缺陷检测方法
CN116295020A (zh) * 2023-05-22 2023-06-23 山东高速工程检测有限公司 一种桥梁病害定位方法及定位装置
CN116295020B (zh) * 2023-05-22 2023-08-08 山东高速工程检测有限公司 一种桥梁病害定位方法及定位装置
CN117054891A (zh) * 2023-10-11 2023-11-14 中煤科工(上海)新能源有限公司 电池寿命的预测方法、预测装置
CN117272245A (zh) * 2023-11-21 2023-12-22 陕西金元新能源有限公司 一种风机齿轮箱温度预测方法、装置、设备和介质
CN117272245B (zh) * 2023-11-21 2024-03-12 陕西金元新能源有限公司 一种风机齿轮箱温度预测方法、装置、设备和介质
CN118034269A (zh) * 2024-01-12 2024-05-14 淮阴工学院 一种用于船舶智能机动的自适应控制方法
CN117710755A (zh) * 2024-02-04 2024-03-15 江苏未来网络集团有限公司 一种基于深度学习的车辆属性识别系统及方法
CN117710755B (zh) * 2024-02-04 2024-05-03 江苏未来网络集团有限公司 一种基于深度学习的车辆属性识别系统及方法
CN118014997A (zh) * 2024-04-08 2024-05-10 湖南联智智能科技有限公司 一种基于改进yolov5的路面病害识别方法
CN118298340A (zh) * 2024-06-06 2024-07-05 北京理工大学长三角研究院(嘉兴) 一种基于先验知识的无人机航拍密集目标检测方法

Also Published As

Publication number Publication date
CN112884760A (zh) 2021-06-01
US20230351573A1 (en) 2023-11-02
CN112884760B (zh) 2023-09-26

Similar Documents

Publication Publication Date Title
WO2022193420A1 (zh) 近水桥梁多类型病害智能检测方法与无人船设备
WO2021142902A1 (zh) 基于DANet的无人机海岸线漂浮垃圾巡检系统
Ma et al. Automatic detection and counting system for pavement cracks based on PCGAN and YOLO-MF
CN112700444B (zh) 基于自注意力与中心点回归模型的桥梁螺栓检测方法
US11948344B2 (en) Method, system, medium, equipment and terminal for inland vessel identification and depth estimation for smart maritime
CN109145747B (zh) 一种水面全景图像语义分割方法
He et al. Automatic damage detection using anchor-free method and unmanned surface vessel
CN111414807A (zh) 一种基于yolo技术的潮水识别与危机预警方法
US11373409B2 (en) Photography system
CN117474914B (zh) 一种基于轻量化神经网络的飞机蒙皮表面缺陷检测方法
Wu et al. Cooperative unmanned surface vehicles and unmanned aerial vehicles platform as a tool for coastal monitoring activities
Yang et al. Automated wall‐climbing robot for concrete construction inspection
CN114266299A (zh) 基于无人机作业的铁路桥梁钢结构缺陷检测方法及系统
CN111008994A (zh) 基于MPSoC的运动目标实时检测跟踪系统及方法
Ma et al. ROV-based binocular vision system for underwater structure crack detection and width measurement
Tan et al. An adaptive crack inspection method for building surface based on BIM, UAV and edge computing
CN112069997B (zh) 一种基于DenseHR-Net的无人机自主着陆目标提取方法及装置
CN116912675B (zh) 一种基于特征迁移的水下目标检测方法及系统
CN116823838B (zh) 高斯先验标签分配与特征解耦的海洋船舶检测方法与系统
CN117649577A (zh) 目标检测模型的训练方法、防外破监测预警方法及装置
CN117011688B (zh) 一种水下结构病害的识别方法、系统及存储介质
Wang et al. Automatic measurement of grid structures displacement through fusion of panoramic camera and laser scanning data
CN117351345A (zh) 一种基于立体视觉的海上目标识别与定位方法
CN114463628A (zh) 一种基于阈值约束的深度学习遥感影像船舰目标识别方法
CN114821494A (zh) 船舶信息匹配方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21931005

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21931005

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21931005

Country of ref document: EP

Kind code of ref document: A1