CN113313107A - Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge - Google Patents

Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge Download PDF

Info

Publication number
CN113313107A
CN113313107A CN202110446806.3A CN202110446806A CN113313107A CN 113313107 A CN113313107 A CN 113313107A CN 202110446806 A CN202110446806 A CN 202110446806A CN 113313107 A CN113313107 A CN 113313107A
Authority
CN
China
Prior art keywords
cable
image
stayed bridge
cylindrical
apparent state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110446806.3A
Other languages
Chinese (zh)
Other versions
CN113313107B (en
Inventor
周朗明
万智
胡帅花
陈晓辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Qiaokang Intelligent Technology Co ltd
Original Assignee
Hunan Qiaokang Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Qiaokang Intelligent Technology Co ltd filed Critical Hunan Qiaokang Intelligent Technology Co ltd
Priority to CN202110446806.3A priority Critical patent/CN113313107B/en
Publication of CN113313107A publication Critical patent/CN113313107A/en
Application granted granted Critical
Publication of CN113313107B publication Critical patent/CN113313107B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/08—Construction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
    • Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping

Abstract

The invention provides a cable-stayed bridge cable surface multi-type disease intelligent detection and identification method, which comprises the following steps: s1: collecting the cable apparent state image information of the cable-stayed bridge through automatic equipment; s2: removing the background in the cable-stayed bridge cable image based on an image preprocessing algorithm, and establishing a cable-stayed bridge cable apparent state image information database; s3: building a cable-stayed bridge cable multi-type disease detection model based on a neural network; s4: training a cable-stayed bridge cable multi-type disease detection model; s5: and predicting the cable-stayed bridge cable image, and determining the disease category and the pixel coordinate information of the cable-stayed bridge cable image. The intelligent detection and identification method for the multiple types of diseases on the cable surface of the cable-stayed bridge, provided by the invention, can quickly and accurately identify the multiple types of diseases on the cable surface of the cable, accurately positions the diseases on the cable-stayed bridge by means of information provided by automatic acquisition equipment, and solves the problems of high cost, high danger, low efficiency, low precision and the like of manual detection of apparent diseases of the cable-stayed bridge.

Description

Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge
Technical Field
The invention relates to the technical field of bridge engineering detection and computer vision, in particular to an intelligent detection and identification method for multiple types of diseases on the cable surface of a cable-stayed bridge.
Background
As a novel bridge type of modern bridges, cable-stayed bridges gradually become the main form of large-span bridges. In China, the large-span cable-stayed bridge has become the most main form of a domestic large bridge due to the characteristics of reasonable load distribution structure, good anti-seismic performance and lower construction cost.
Apparent corrosion damage of a cable-stayed bridge is a main cause of accidents of most cable-stayed bridges, particularly rust corrosion, wire breakage failure and other diseases of steel wires in a stay cable, and in detection and maintenance of the cable-stayed bridge, apparent detection and protection problems of the cable are always taken as important detection contents.
The conventional cable-stayed bridge cable inspection method comprises two methods: manual detection method and hanging basket detection method. The manual detection method is characterized in that the surface of the stay cable is inspected through visual inspection by bridge maintenance personnel or through carrying detection equipment so as to find and feed back the damage of the surface of the stay cable, and the method has strong subjectivity, low detection efficiency and certain danger; as for a hanging basket detection and maintenance method, high-altitude operation is required, the cost is high, the efficiency is low, the danger is high, and the normal traffic is influenced.
A team of Chongqing university Zhongji professor designs a cable surface defect detection system, extracts defect edges by using a canny operator, performs feature recognition on images by using a fuzzy set method, and finally performs defect classification based on the gray scale and shape features of the images, so that cable surface attachments and defects cannot be distinguished. A Xufengyu team at university in southeast designs a cable surface damage detection system, extracts a defect outline by using an edge detection algorithm, and judges the defect based on the area of the defect without classifying the defect. The two sets of defect detection systems are used for detecting the surface defects of the cables based on the traditional image processing method, and classification of the defects, surface attachments and defect distinction are required to be optimized and perfected.
In addition, cable-stayed bridge cable image acquisition is carried out outdoors, and due to the influence of uneven illumination and complex background, the acquired image has the situations of exposure, shadow, unobvious background and cable gray scale difference and the like. In order to improve the precision of disease detection and identification, the invention firstly preprocesses the cable-stayed bridge cable image before deep learning network training and detection, cuts the cable-stayed bridge cable target and the background, and adjusts the image contrast through histogram equalization; the cable-stay bridge cable is the cylinder, can take place the distortion of surface texture at the imaging in-process, and the surface area of curved surface can compress the image when shooing is unfavorable for the detection and the discernment of disease.
Deep learning is a method based on characterization learning of data, and combines low-level features to form a more abstract high-level representation attribute category or feature to find a distributed feature representation of the data. In the research of image classification, most of the feature extraction processes are designed manually, the bottom-layer features of the image are obtained through shallow learning, and a great semantic gap exists between the bottom-layer features and the high-level subjects of the image. The deep learning completely learns the hierarchical structural characteristics of the image from the training data by using the set network structure, and can extract the abstract characteristics closer to the high-level semantics of the image, so the expression of the image recognition is far superior to that of the traditional method.
The noun explains:
and (3) mean shift algorithm: the method is a non-parametric statistical iterative algorithm for seeking a value point in a sample data set, and is also an iterative algorithm for non-parametric kernel density estimation. The core of the method is to cluster sample points in an image feature space, wherein the sample points converge to a point with a zero value, namely a module value point, along the ascending direction of the probability density gradient.
Flood filling algorithm: also known as seed filling, is an algorithm that extracts several connected points from a region to distinguish them from other adjacent regions.
Canny operator: the method is an edge detection algorithm and has good signal-to-noise ratio and detection precision. The principle is to perform gaussian smoothing on the input image to reduce the error rate. And calculating the gradient amplitude and direction to estimate the edge strength and direction of each point, and performing non-maximum suppression on the gradient amplitude according to the gradient direction. Finally, the edges are detected and connected by double threshold processing.
Straight Line Segment Detector (LSD) algorithm: the method is a line segment detection algorithm, and can obtain a line segment detection result with sub-pixel level precision in a short time. Calculating the gradient size and direction of all points in the image, taking the points with small gradient direction change and adjacent points as a connected domain, judging whether the points need to be disconnected according to rules according to the rectangularity of each domain to form a plurality of domains with larger rectangularity, finally improving and screening all the generated domains, and reserving the domains meeting the conditions, namely the final straight line detection result. The algorithm has the advantages of high detection speed, no need of parameter adjustment and utilization of an error control method to improve the accuracy of linear detection.
Histogram equalization algorithm: the method is an image enhancement algorithm, and is used for equalizing the gray value of an original image, reducing the gray level with less pixels in the image, and widening the gray level with more pixels in the image, so that the corresponding histogram of the image is in a uniform distribution form. Therefore, the contrast is increased, the image details are clear, and the image space enhancement is realized.
Cylindrical back projection: is the process of projecting a particular viewing area of the cylindrical surface onto the cylinder's tangential plane.
Disclosure of Invention
Based on the background current situation, the invention aims to provide an intelligent detection and identification method for multiple types of diseases on the surface of a cable-stayed bridge, which adopts automatic equipment to realize 100% nondestructive detection on the surface condition of a cable-stayed protection layer, realizes intelligent detection and identification of the multiple types of diseases on the surface of the cable-stayed bridge by means of an image preprocessing algorithm and a deep learning algorithm, corrects a cylindrical cable image by a combined cylindrical surface expansion algorithm, projects a cylindrical curved surface to a two-dimensional plane, and facilitates the detection and identification of the diseases on the surface of the cable-stayed bridge; the problems of disease classification, cable surface attachment and disease distinguishing and the like are solved, the working efficiency and safety are greatly improved, the maintenance cost is reduced, the application prospect is good, and meanwhile great economic value and social value are created.
The invention provides space geometric information and image information based on camera calibration.
A cable-stayed bridge cable surface multi-type disease intelligent detection and identification method comprises the following steps:
s1: acquiring an apparent state image of a cable-stayed bridge, wherein the apparent state image of the cable-stayed bridge is used for representing a complete apparent state image of the cable-stayed bridge at 360 degrees; the cable-stayed bridge cable apparent state image comprises at least four images acquired in different directions;
s2: removing a background in the cable-stayed bridge cable image based on an image preprocessing algorithm, and correcting the cylindrical cable image by adopting a cylindrical surface expansion algorithm to obtain a cylindrical cable image; manually marking the diseases on the cylindrical cable images, and establishing an image information database of the apparent state of the cable-stayed bridge;
s3: dividing an image information database of the apparent state of the cable-stayed bridge cable into a training set and a verification set, inputting the training set into a neural network for training, and training to obtain a multi-type disease detection model of the cable-stayed bridge cable;
s4: the method comprises the steps that a multi-type disease detection model of a cable-stayed bridge cable is evaluated on a verification set, and the multi-type disease detection model of the cable-stayed bridge cable with the highest precision is selected as a final multi-type disease detection model of the cable-stayed bridge cable;
s5: and (4) acquiring an apparent state image of the cable-stayed bridge cable to be detected, then obtaining a cylindrical cable image of the cable-stayed bridge cable to be detected according to the step S2, and inputting the cylindrical cable image of the cable-stayed bridge cable to be detected into a final multi-type disease detection model of the cable-stayed bridge cable to obtain the type and pixel coordinate position information of the disease.
In a further improvement, in step S2, the defect on the cylindrical cable image includes scratch, damage, and peeling off.
Further improvement, the image preprocessing algorithm based background removal in the cable-stayed bridge cable image and the establishment of the cable-stayed bridge cable apparent state image information database specifically comprise the following steps:
s2.1: performing color dithering, zooming and Gaussian noise addition on the cable-stayed bridge cable apparent state image information acquired in the step S1 to perform data enhancement, and increasing the number of apparent state images;
s2.2: pre-dividing the apparent state image obtained in the step S2.1 by adopting a mean shift and flood filling method, and dividing different areas;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator;
s2.4: detecting a straight line of the image with the edge extracted by utilizing a straight line segment detector algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary;
s2.6: according to the cable boundary extracted in the S2.5, removing the background outside the cable, and filling the background with white to obtain a cylindrical cable image;
s2.7: adjusting the brightness of the image through a histogram matching algorithm;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
s2.9: labeling the corrected cable-stayed bridge cable image by using labelImg software, labeling the category of the disease and the pixel coordinate position information to obtain a positive sample, searching for a false disease which is easy to be confused and taking the false disease as a negative sample, and labeling no information on the negative sample;
s2.10: and (3) forming an image information database of the cable apparent state of the cable-stayed bridge by using the positive and negative samples, dividing the database into a training set and a verification set according to the ratio of 9:1 by random sampling, wherein the training set is used for training a model, and the verification set is used for evaluating the quality of the model.
In a further improvement, the cylindrical surface expansion algorithm for correcting the cylindrical cable image comprises the following steps:
and the cylindrical cable image is an ideal cylindrical image, is expanded along the bus direction, and is projected onto a two-dimensional plane by adopting cylindrical back projection to form a two-dimensional image. Around the center O of the cylinderwEstablishing a three-dimensional coordinate system for the origin of coordinates, along the optical center O of the cameracIn the direction ZwAxis perpendicular to OcOwIn the horizontal direction of the plane XwAxis, perpendicular to direction YwA shaft; taking any point on the surface of the cylinder image
Figure BDA0003036927330000031
Corresponding to points on the projected two-dimensional image
Figure BDA0003036927330000032
Is set as the radius r of the cylinder and the optical center O of the cameracDistance f to the projected two-dimensional image, camera optical center OcTo the center O of the cylinderwIs g; the calculation formula for point P is as follows:
Figure BDA0003036927330000033
angle theta is the projection of point P on the cylindrical target onto XwZwAxial plane and XwThe axes forming an angle, d being the points P to X on the image of the cylinderwZwThe vertical distance of the axial plane; setting point P at XwZwThe projection of the axial plane is point C, which is projected to ZwThe axes being points A, ZwShaft and OcOwOn the same straight line, therefore CA is perpendicular to OcOw(ii) a Optical center of camera OcTo the center O of the cylinderwThe connecting line of the two-dimensional coordinate system is perpendicular to the projected two-dimensional plane and intersects at a point a, and a is taken as an original point, the horizontal direction is an X axis, and the vertical direction is a Y axis to establish a two-dimensional coordinate system; setting the projection of the point P' on the X axis as a point c, ca is perpendicular to OcOw(ii) a According to the principle of similar triangle, under the premise that the center of the cylindrical target is positioned at the center of the image, the following results are obtained：
Figure BDA0003036927330000041
Figure BDA0003036927330000042
Where m is the height of the cylindrical object, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, and g is the center O of the cylindrical objectwTo the optical center of the camera OcThe distance of (d);
the coordinates of the two-dimensional point p' are derived from equations (2) (3):
Figure BDA0003036927330000043
the cylindrical cable image is mapped into a two-dimensional plane according to equation (4), which achieves the correction.
In a further improvement, in step S1, the cable-stayed bridge cable appearance state image information is collected by an automatic collecting device, wherein the automatic collecting device has 4 cameras, and the 4 cameras shoot the complete appearance state image of the cylindrical cable-stayed cable at 360 degrees.
Drawings
FIG. 1 is a flow chart of a method for intelligently detecting and identifying multiple types of diseases on the surface of a cable-stayed bridge according to the present invention;
FIG. 2 is a diagram illustrating the effect of the background removal algorithm provided by the present invention; (a) an original image; (b) pre-dividing the image; (c) an edge detection image; (d) detecting an image of the line segment; (e) fitting an image to the boundary; (f) background removal and histogram equalization of the image.
FIG. 3 is a cylindrical object imaging geometry provided by the present invention;
FIG. 4 is a diagram illustrating a calibration result of a cylindrical surface according to the present invention; (a) a photographed cylindrical image; (b) a corrected cylindrical image;
FIG. 5 is a diagram of a model network architecture provided by the present invention;
FIG. 6 is a flow chart of model training provided by the present invention;
FIG. 7 shows the results of detecting surface defects of cable-stayed bridges according to the present invention; (a) detecting a scratch; (b) a damage detection result; (c) and (5) peeling detection results.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the following will describe embodiments of the present invention in further detail with reference to the accompanying drawings, and the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Other embodiments obtained by persons of ordinary skill in the art without inventive faculty based on the embodiments of the present invention are within the scope of the present invention.
As shown in fig. 1, the method for intelligently detecting and identifying multiple types of diseases on the surface of a cable-stayed bridge provided by the invention comprises the following steps:
s1: collecting the cable apparent state image information of the cable-stayed bridge through automatic equipment;
in the task of detecting the apparent state of the cable-stayed bridge cable, the establishment of an apparent state database is very important, and the quality of the database has great influence on the effect of a detection algorithm. The method has the advantages that the number of the image data of the apparent state of the cable-stayed bridge cable acquired by manual shooting is small, the cost is high, and the image data are not complete, so that the method adopts automatic acquisition equipment to automatically acquire the image data of the apparent state of the cable-stayed bridge cable in all directions, the automatic acquisition equipment is provided with 4 cameras, the 4 cameras can shoot the complete image of the apparent state of the cylindrical cable-stayed cable at 360 degrees, and the acquired image is transmitted back to ground storage equipment through a wireless network;
s2: removing a background in a cable image of the cable-stayed bridge based on an image preprocessing algorithm, correcting the cylindrical cable image by adopting a cylindrical surface expansion algorithm, and establishing an apparent state image information database of the cable-stayed bridge;
on the basis of the above technical solution, the step S2 specifically includes:
s2.1: and (3) performing data enhancement such as color dithering, scaling and Gaussian noise addition on the apparent state image information acquired in the step (S1), and increasing the number of the apparent state images, wherein the variation range of the color dithering number is [ -0.1, +0.1], the scaling size is 0.25, 0.5, 0.75, 1.25, 1.5, 1.75 and 2.0 times of the original image, and the Gaussian noise is a random number value with 0.2 as a mean value and 0.3 as a standard deviation.
S2.2: pre-dividing the image obtained in the step S2.1 by adopting a mean shift and flood filling method, dividing different areas, firstly dividing the image by using a mean shift algorithm, and then coloring the different divided areas by using a flood filling algorithm;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator, firstly converting the pre-segmented image into a gray-scale image, and then extracting edges from the gray-scale image by using the edge detection Canny operator to obtain a binary edge extraction result image;
s2.4: detecting a straight line of the image after the edge is extracted by utilizing a straight Line Segment Detector (LSD) algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary; the cable is a cylinder, the boundary of the acquired cable image is relatively parallel, the slope of the straight line detected in S2.4 is calculated, and parallel line segments are screened out. Calculating the distance between the parallel line segments, and screening out the cable boundary under the constraint that the distance between two edges of the cable is in a certain range according to the fixed distance between the camera and the cable; according to statistics, the distance d between two sides of the cable in this embodiment ranges from
Figure BDA0003036927330000051
w is the width of the acquired image.
S2.6: according to the cable boundary extracted in S2.5, removing the background outside the cable, and filling the background with white;
s2.7: the contrast of the image is adjusted through a histogram matching algorithm, and a background removing algorithm effect graph is shown in FIG. 2;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
the cylindrical cable image is an ideal cylindrical image, is unfolded along the generatrix direction, and adopts cylindrical back projection to project the cylindrical cable imageThe cylindrical surface observation area shot by the camera is projected to a two-dimensional plane to form a two-dimensional image. Around the center O of the cylinderwEstablishing a three-dimensional coordinate system for the origin of coordinates, along the optical center O of the cameracIn the direction ZwAxis perpendicular to OcOwIn the horizontal direction of the plane XwAxis, perpendicular to direction YwA shaft; taking any point on the surface of the cylinder image
Figure BDA0003036927330000061
Corresponding to points on the projected two-dimensional image
Figure BDA0003036927330000062
Is set as the radius r of the cylinder and the optical center O of the cameracDistance f to the projected two-dimensional image, camera optical center OcTo the center O of the cylinderwIs g; the calculation formula for point P is as follows:
Figure BDA0003036927330000063
angle theta is the projection of point P on the cylindrical target onto XwZwAxial plane and XwThe axes forming an angle, d being the points P to X on the image of the cylinderwZwThe vertical distance of the axial plane; setting point P at XwZwThe projection of the axial plane is point C, which is projected to ZwThe axes being points A, ZwShaft and OcOwOn the same straight line, therefore CA is perpendicular to OcOw(ii) a Optical center of camera OcTo the center O of the cylinderwThe connecting line of the two-dimensional coordinate system is perpendicular to the projected two-dimensional plane and intersects at a point a, and a is taken as an original point, the horizontal direction is an X axis, and the vertical direction is a Y axis to establish a two-dimensional coordinate system; setting the projection of the point P' on the X axis as a point c, ca is perpendicular to OcOw(ii) a According to the principle of similar triangles, on the premise that the center of the cylindrical target is positioned at the center of the image, the following results are obtained:
Figure BDA0003036927330000064
Figure BDA0003036927330000065
where m is the height of the cylindrical object, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, and g is the center O of the cylindrical objectwTo the optical center of the camera OcThe distance of (d);
the coordinates of the two-dimensional point p' are derived from equations (2) (3):
Figure BDA0003036927330000066
the cylindrical cable image is mapped into a two-dimensional plane according to equation (4), which achieves the correction. The photographed lenticular image is corrected by the lenticular expansion algorithm as shown in fig. 4(a), and the corrected lenticular image is shown in fig. 4 (b).
S2.9: labeling the corrected cable-stayed bridge cable image by using labelImg software, labeling the type of the disease and pixel coordinate position information to obtain a positive sample, specifically labeling to draw a minimum circumscribed rectangle at the disease position, inputting the type of the disease, generating an xml file with the same name as the image name, wherein the xml file comprises the type of the disease and the pixel coordinate information, and analyzing the xml file by a program to generate a txt file required by model training; searching confusable false diseases as negative samples, wherein the negative samples greatly increase the robustness of the algorithm, and the negative samples do not need to be marked with any information to generate an empty txt file;
s2.10: forming a cable-stayed bridge cable database by using the positive and negative samples, wherein the database contains 27070 images, and the negative sample images 10200 are in a JPG format; dividing a database into a training set and a verification set according to a ratio of 9:1 by random sampling, wherein the training set is used for training a model, and the verification set is used for evaluating the quality of the model;
the cable surface of the cable-stayed bridge is subjected to various diseases including but not limited to scratches, damages, peeling and line dropping;
s3: building a cable-stayed bridge cable multi-type disease detection model based on a neural network YOLOV 5;
fig. 5 is a diagram of a model network structure, in which an input image continuously extracts features through a backbone network, a shallow feature extracts geometric texture information, a deep feature extracts abstract semantic information, the network structure transversely connects the shallow feature, a middle feature and the deep feature through three branches, the deep feature, the middle feature and the shallow feature are fused through a fusion channel, and finally the three shallow feature, the middle feature and the deep feature fused with other feature information are output, and an output result includes category information of a disease and bounding box pixel coordinate information.
S4: training a multi-type disease detection model of a cable-stayed bridge cable, and evaluating the model on a verification set to obtain a model with the highest precision;
FIG. 6 is a model training flow chart, wherein training data is input, and hyper-parameters are set, wherein the hyper-parameters are specifically set as follows: the prior frame size is obtained by a k-means clustering algorithm, specifically, 89,40,49,82,149,46,111,78,48,184,81,118,227,72,58,335,158,122, 105,194, 118,337, 69,601, 219,212, 161,550, 339, 345;
the batch size (batch) is set to be 32, a random gradient descent (SGD) algorithm of Momentum (Momentum) is adopted, the Momentum value is set to be 0.93, and the weight decay coefficient (weight decay) is set to be 0.0009; the initial learning rate was set to 0.013, and in the first 2000 iterations, the learning rate was linearly increased from 0 to 0.013, and then the learning rate was varied as a cosine function in each cycle with a total number of iterations of 39000, with a period of 4000 iterations.
According to the method, the model is trained on the ImageNet data set training sample, and then the trained network model parameters are initialized to be the parameters of the cable-stayed bridge cable multi-type disease detection model, so that the network model training convergence speed and the model performance can be improved.
Loading a pre-training model for network training, predicting disease categories and a surrounding frame thereof by extracting features through a model network, calculating a loss function value, updating model parameters based on a back propagation algorithm, judging whether the model training is finished or not based on the evaluation precision of the model on a verification set, finishing the training if the evaluation precision converges to a certain value, storing the model parameters at the moment, finishing the model training, and otherwise, continuing the training of the model until the evaluation precision converges.
S5: predicting an image of a cable-stayed bridge cable after background removal through image preprocessing, determining the disease category and pixel coordinate information of the image, removing the background of the image acquired by an automatic device by using a preprocessing algorithm in the step S2, correcting the image of the cylindrical cable through a cylindrical surface unfolding algorithm, sending the corrected image into a model trained in the step S4 for prediction, analyzing an output result, and obtaining the disease category and pixel coordinate position information;
the above-described series of detailed descriptions are merely specific to possible embodiments of the present invention, and they are not intended to limit the scope of the present invention, and various changes made without departing from the gist of the present invention within the knowledge of those skilled in the art are included in the scope of the present invention.

Claims (5)

1. The cable-stayed bridge cable surface multi-type disease intelligent detection and identification method is characterized by comprising the following steps:
s1: acquiring an apparent state image of a cable-stayed bridge, wherein the apparent state image of the cable-stayed bridge is used for representing a complete apparent state image of the cable-stayed bridge at 360 degrees; the cable-stayed bridge cable apparent state image comprises at least four images acquired in different directions;
s2: removing a background in the cable-stayed bridge cable image based on an image preprocessing algorithm, and correcting the cylindrical cable image by adopting a cylindrical surface expansion algorithm to obtain a cylindrical cable image; manually marking the diseases on the cylindrical cable images, and establishing an image information database of the apparent state of the cable-stayed bridge;
s3: dividing an image information database of the apparent state of the cable-stayed bridge cable into a training set and a verification set, inputting the training set into a neural network for training, and training to obtain a multi-type disease detection model of the cable-stayed bridge cable;
s4: the method comprises the steps that a multi-type disease detection model of a cable-stayed bridge cable is evaluated on a verification set, and the multi-type disease detection model of the cable-stayed bridge cable with the highest precision is selected as a final multi-type disease detection model of the cable-stayed bridge cable;
s5: and (4) acquiring an apparent state image of the cable-stayed bridge cable to be detected, then obtaining a cylindrical cable image of the cable-stayed bridge cable to be detected according to the step S2, and inputting the cylindrical cable image of the cable-stayed bridge cable to be detected into a final multi-type disease detection model of the cable-stayed bridge cable to obtain the type and pixel coordinate position information of the disease.
2. The method for intelligently detecting and identifying multiple types of defects on the surface of a cable-stayed bridge cable according to claim 1, wherein in the step S2, the defects on the cylindrical cable image include scratches, damages and peeling-off.
3. The method for intelligently detecting and identifying the multiple types of diseases on the surface of the cable-stayed bridge cable according to claim 1, wherein the background in the cable-stayed bridge cable image is removed based on an image preprocessing algorithm, and an image information database of the apparent state of the cable-stayed bridge cable is established, and the method specifically comprises the following steps:
s2.1: performing color dithering, zooming and Gaussian noise addition on the cable-stayed bridge cable apparent state image information acquired in the step S1 to perform data enhancement, and increasing the number of apparent state images;
s2.2: pre-dividing the apparent state image obtained in the step S2.1 by adopting a mean shift and flood filling method, and dividing different areas;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator;
s2.4: detecting a straight line of the image with the edge extracted by utilizing a straight line segment detector algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary;
s2.6: according to the cable boundary extracted in the S2.5, removing the background outside the cable, and filling the background with white to obtain a cylindrical cable image;
s2.7: adjusting the brightness of the image through a histogram matching algorithm;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
s2.9: labeling the corrected cable-stayed bridge cable image by using labelImg software, labeling the category of the disease and the pixel coordinate position information to obtain a positive sample, searching for a false disease which is easy to be confused and taking the false disease as a negative sample, and labeling no information on the negative sample;
s2.10: and (3) forming an image information database of the cable apparent state of the cable-stayed bridge by using the positive and negative samples, dividing the database into a training set and a verification set according to the ratio of 9:1 by random sampling, wherein the training set is used for training a model, and the verification set is used for evaluating the quality of the model.
4. The method for intelligently detecting and identifying multiple types of diseases on the surface of a cable-stayed bridge cable according to claim 1 or 3, wherein the step of correcting the cylindrical cable image by using a cylindrical surface expansion algorithm comprises the following steps:
setting a cylindrical cable image as an ideal cylindrical image, expanding the cylindrical cable image along the bus direction, and projecting a cylindrical surface observation area shot by a camera onto a two-dimensional plane by adopting cylindrical back projection to form a two-dimensional image; around the center O of the cylinderwEstablishing a three-dimensional coordinate system for the origin of coordinates, along the optical center O of the cameracIn the direction ZwAxis perpendicular to OcOwIn the horizontal direction of the plane XwAxis, perpendicular to direction YwA shaft; taking any point on the surface of the cylinder image
Figure FDA0003036927320000021
Corresponding to points on the projected two-dimensional image
Figure FDA0003036927320000022
Is set as the radius r of the cylinder and the optical center O of the cameracDistance f to the projected two-dimensional image, camera optical center OcTo the center O of the cylinderwIs g; the calculation formula for point P is as follows:
Figure FDA0003036927320000023
angle theta is the projection of point P on the cylindrical target onto XwZwAxial plane and XwThe axes forming an angle, d being the points P to X on the image of the cylinderwZwThe vertical distance of the axial plane; setting point P at XwZwThe projection of the axial plane is point C, which is projected to ZwThe axes being points A, ZwShaft and OcOwOn the same straight line, therefore CA is perpendicular to OcOw(ii) a Optical center of camera OcTo the center O of the cylinderwThe connecting line of the two-dimensional coordinate system is perpendicular to the projected two-dimensional plane and intersects at a point a, and a is taken as an original point, the horizontal direction is an X axis, and the vertical direction is a Y axis to establish a two-dimensional coordinate system; setting the projection of the point P' on the X axis as a point c, ca is perpendicular to OcOw(ii) a According to the principle of similar triangles, on the premise that the center of the cylindrical target is positioned at the center of the image, the following results are obtained:
Figure FDA0003036927320000024
Figure FDA0003036927320000025
where m is the height of the cylindrical object, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, and g is the center O of the cylindrical objectwTo the optical center of the camera OcThe distance of (d);
the coordinates of the two-dimensional point p' are derived from equations (2) (3):
Figure FDA0003036927320000026
the cylindrical cable image is mapped into a two-dimensional plane according to equation (4), which achieves the correction.
5. The method for intelligently detecting and identifying the multiple types of diseases on the surface of the cable-stayed bridge cable according to claim 1 or 3, wherein in the step S1, the information of the image of the apparent state of the cable-stayed bridge cable is collected by an automatic device, the automatic device comprises 4 cameras, and the 4 cameras shoot the complete image of the apparent state of the cylindrical cable-stayed cable at 360 degrees.
CN202110446806.3A 2021-04-25 2021-04-25 Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge Active CN113313107B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110446806.3A CN113313107B (en) 2021-04-25 2021-04-25 Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110446806.3A CN113313107B (en) 2021-04-25 2021-04-25 Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge

Publications (2)

Publication Number Publication Date
CN113313107A true CN113313107A (en) 2021-08-27
CN113313107B CN113313107B (en) 2023-08-15

Family

ID=77371085

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110446806.3A Active CN113313107B (en) 2021-04-25 2021-04-25 Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge

Country Status (1)

Country Link
CN (1) CN113313107B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113960069A (en) * 2021-10-22 2022-01-21 中铁二十二局集团第五工程有限公司 Method for establishing surface morphology of cable through laser line scanning
CN114280061A (en) * 2021-12-27 2022-04-05 交通运输部公路科学研究所 Observation method for technical conditions of cable-stayed bridge cable beam anchoring area and monitoring window
CN115272276A (en) * 2022-08-12 2022-11-01 东南大学 Suspension bridge main cable subsurface disease identification method and device based on infrared light camera shooting

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101169380A (en) * 2007-10-31 2008-04-30 重庆大学 Bridge cable surface damage dynamic detection method and device
CN106856003A (en) * 2016-12-31 2017-06-16 南京理工大学 The expansion bearing calibration of shaft-like workpiece side surface defects detection image
CN107796835A (en) * 2017-10-20 2018-03-13 北京航空航天大学 A kind of X ray cylinder 3 D cone-beam computer tomography method and device
CN111127399A (en) * 2019-11-28 2020-05-08 东南大学 Underwater pier disease identification method based on deep learning and sonar imaging
CN111191714A (en) * 2019-12-28 2020-05-22 浙江大学 Intelligent identification method for bridge appearance damage diseases
CN111985499A (en) * 2020-07-23 2020-11-24 东南大学 High-precision bridge apparent disease identification method based on computer vision

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101169380A (en) * 2007-10-31 2008-04-30 重庆大学 Bridge cable surface damage dynamic detection method and device
CN106856003A (en) * 2016-12-31 2017-06-16 南京理工大学 The expansion bearing calibration of shaft-like workpiece side surface defects detection image
CN107796835A (en) * 2017-10-20 2018-03-13 北京航空航天大学 A kind of X ray cylinder 3 D cone-beam computer tomography method and device
CN111127399A (en) * 2019-11-28 2020-05-08 东南大学 Underwater pier disease identification method based on deep learning and sonar imaging
CN111191714A (en) * 2019-12-28 2020-05-22 浙江大学 Intelligent identification method for bridge appearance damage diseases
CN111985499A (en) * 2020-07-23 2020-11-24 东南大学 High-precision bridge apparent disease identification method based on computer vision

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113960069A (en) * 2021-10-22 2022-01-21 中铁二十二局集团第五工程有限公司 Method for establishing surface morphology of cable through laser line scanning
CN113960069B (en) * 2021-10-22 2024-03-19 中铁二十二局集团第五工程有限公司 Method for establishing cable surface morphology through laser line scanning
CN114280061A (en) * 2021-12-27 2022-04-05 交通运输部公路科学研究所 Observation method for technical conditions of cable-stayed bridge cable beam anchoring area and monitoring window
CN114280061B (en) * 2021-12-27 2022-10-14 交通运输部公路科学研究所 Observation method for technical conditions of cable-stayed bridge cable beam anchoring area and monitoring window
CN115272276A (en) * 2022-08-12 2022-11-01 东南大学 Suspension bridge main cable subsurface disease identification method and device based on infrared light camera shooting

Also Published As

Publication number Publication date
CN113313107B (en) 2023-08-15

Similar Documents

Publication Publication Date Title
US11221107B2 (en) Method for leakage detection of underground pipeline corridor based on dynamic infrared thermal image processing
CN111444821B (en) Automatic identification method for urban road signs
CN112380952B (en) Power equipment infrared image real-time detection and identification method based on artificial intelligence
CN108615226B (en) Image defogging method based on generation type countermeasure network
CN113313107B (en) Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge
CN110148130B (en) Method and device for detecting part defects
CN110264448B (en) Insulator fault detection method based on machine vision
CN110309781B (en) House damage remote sensing identification method based on multi-scale spectrum texture self-adaptive fusion
CN111814686A (en) Vision-based power transmission line identification and foreign matter invasion online detection method
CN111027446B (en) Coastline automatic extraction method of high-resolution image
CN114663346A (en) Strip steel surface defect detection method based on improved YOLOv5 network
CN112330593A (en) Building surface crack detection method based on deep learning network
CN112560675B (en) Bird visual target detection method combining YOLO and rotation-fusion strategy
CN114749342B (en) Lithium battery pole piece coating defect identification method, device and medium
CN109685045A (en) A kind of Moving Targets Based on Video Streams tracking and system
CN114926407A (en) Steel surface defect detection system based on deep learning
CN115082744B (en) Artificial intelligence-based solar heat collection efficiency analysis method and system
CN109584206B (en) Method for synthesizing training sample of neural network in part surface flaw detection
CN108665468B (en) Device and method for extracting tangent tower insulator string
CN116563262A (en) Building crack detection algorithm based on multiple modes
CN113435407A (en) Small target identification method and device for power transmission system
CN115841633A (en) Power tower and power line associated correction power tower and power line detection method
CN115222884A (en) Space object analysis and modeling optimization method based on artificial intelligence
CN110348307B (en) Path edge identification method and system for crane metal structure climbing robot
CN111178405A (en) Similar object identification method fusing multiple neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant