CN114862892A - Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics - Google Patents
Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics Download PDFInfo
- Publication number
- CN114862892A CN114862892A CN202210601919.0A CN202210601919A CN114862892A CN 114862892 A CN114862892 A CN 114862892A CN 202210601919 A CN202210601919 A CN 202210601919A CN 114862892 A CN114862892 A CN 114862892A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- edge
- data
- pavement
- crack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000003708 edge detection Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 12
- 238000005286 illumination Methods 0.000 claims abstract description 10
- 238000012360 testing method Methods 0.000 claims abstract description 7
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000003287 optical effect Effects 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 230000005856 abnormality Effects 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 4
- 230000000717 retained effect Effects 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 abstract description 7
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 20
- 238000012545 processing Methods 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000000877 morphologic effect Effects 0.000 description 4
- 238000012805 post-processing Methods 0.000 description 4
- 239000010426 asphalt Substances 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000004895 liquid chromatography mass spectrometry Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000004568 cement Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G06T5/70—
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Processing (AREA)
Abstract
The invention provides a two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics, which comprises the following steps: reading in three-dimensional pavement data/two-dimensional pavement data acquired by a pavement acquisition system; preprocessing data; removing uneven illumination information of three-dimensional pavement data attitude fluctuation/two-dimensional pavement data; acquiring edge enhancement data; integrally nesting edge detection deep network training, and acquiring a test data edge probability graph according to a training model; s6, converting the edge probability map to obtain strong edge information of the road surface data; s7, selecting and connecting strong edges; and S8, extracting crack attribute information of the two-dimensional/three-dimensional road surface. The method integrates the frequency component characteristics of the cross section of the pavement and the multi-scale edge characteristics acquired by the deep learning network, enhances the edges of the pavement cracks, is beneficial to accurately extracting two-dimensional and three-dimensional pavement data cracks, and is suitable for automatic crack identification tasks of various typical pavements acquired by a two-dimensional and three-dimensional data acquisition system.
Description
Technical Field
The invention relates to the technical field of automatic detection and identification of pavement damage, in particular to a two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics.
Background
With the development of highway pavement damage measuring equipment and automatic detection technology, the vehicle-mounted highway pavement measuring system can acquire more and more two-dimensional and three-dimensional pavement data including data of different pavement backgrounds and different crack types. The two-dimensional optical road surface data acquisition equipment has the characteristics of economy and mature technology, is suitable for introducing a typical machine learning method, and is easily influenced by illumination, shadow, road surface tire wear marks, oil stains and the like in practical application. The line scanning three-dimensional road surface detection method can overcome the interference of uneven illumination shadow in the actual road surface detection project, can acquire three-dimensional information of road surface damage, and is the current development trend. However, the high-precision three-dimensional road data contains severe vehicle-carrying attitude fluctuation information in an actual dynamic acquisition environment, and the three-dimensional data contains road surface damage, road surface texture, sign lines and elevation information of other road surface targets. Compared with the rapid development of a two-dimensional optical road surface damage detection algorithm, the current three-dimensional road surface damage detection technology is relatively lagged. The cracks are the most common damage types in the road surface damage and are the most concerned detection indexes in the application of the actual road surface maintenance detection engineering. Because two-dimensional pavement data and three-dimensional pavement data have certain advantages in actual highway detection engineering application, the two-dimensional pavement data and the three-dimensional pavement data play a role in specific detection engineering at present. However, the current methods for detecting cracks of two-dimensional road surfaces and three-dimensional road surfaces are independent from each other, and a method capable of simultaneously processing crack detection of two-dimensional road surface data and three-dimensional road surface data is lacked.
Due to the characteristics of massive large data of two-dimensional and three-dimensional pavement data in the practical pavement crack detection engineering application, the method for quickly and automatically detecting the crack position and the attribute information from the massive pavement data by using a machine learning method has important theoretical and application values. Because the two-dimensional optical pavement data more accords with the data set requirement of a computer vision machine learning model, the two-dimensional optical pavement machine learning deep learning method is widely researched and practically applied. However, machine learning is difficult to obtain a good effect in the application of directly scanning the crack detection of the three-dimensional pavement data on line, the interference of factors such as driving postures and deformation diseases is included in the three-dimensional pavement data scanned on one side, the robustness of an actual crack detection task on the method is high, and the method is required to be suitable for different types of cracks and different pavement backgrounds; on the other hand, because the difference of crack characteristics in the three-dimensional data and the two-dimensional data is large, the three-dimensional pavement marking is generally more difficult to obtain, and the traditional supervision machine learning method and even the deep learning method have limited applicability to different pavement cracks with different data.
It should be noted that the above background description is only for the convenience of clear and complete description of the technical solutions of the present application and for the understanding of those skilled in the art. Such solutions are not considered to be known to the person skilled in the art merely because they have been set forth in the background section of the present application.
Disclosure of Invention
The purpose of the invention is: aiming at the defects in the background technology, the two-dimensional and three-dimensional pavement crack automatic detection method based on the enhanced depth edge characteristics is provided, so that the continuous edge characteristics of cracks expressed in pavement data are fully enhanced and utilized, and the method can be suitable for crack detection application of extraction of various typical pavement cracks of two-dimensional and three-dimensional pavement data.
In order to achieve the purpose, the invention provides a two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics, which comprises the following steps:
s1, reading in three-dimensional pavement data/two-dimensional pavement data acquired by a pavement acquisition system;
s2, preprocessing data;
s3, removing uneven illumination information of the three-dimensional road surface data attitude fluctuation/two-dimensional road surface data;
s4, acquiring edge enhancement data;
s5, performing integral nested edge detection deep network training, and acquiring a test data edge probability graph according to a training model;
s6, converting the edge probability map to obtain strong edge information of the road surface data;
s7, selecting and connecting strong edges;
and S8, extracting crack attribute information of the two-dimensional/three-dimensional road surface.
Furthermore, the two-dimensional pavement data is pavement optical data acquired by a vehicle-mounted two-dimensional camera, and the three-dimensional pavement data is acquired by acquiring a series of pavement section profiles along the measuring direction by using a line scanning three-dimensional measuring sensor and splicing the series of pavement section profiles.
Further, it is characterized in that in S2, abnormal values generated by system abnormality and environmental abnormality are corrected, the cross-sectional profile is converted from image space to object space by a calibration file, cross-sectional data of the object space is acquired, and system errors in the measurement system are corrected.
Further, in S3, for the acquired three-dimensional road surface data/two-dimensional road surface data, each cross section is subjected to component decomposition, the influence of low-frequency components is removed, and the high-frequency edge characteristics of the crack are enhanced.
Further, in S4, the high-frequency components of the cross sections are spliced to form high-frequency component data with enhanced edge characteristics, and the data with enhanced edge characteristics are uniformly converted into a gray scale map with values distributed in the range of 0 to 255.
Further, in the step S5, the data obtained in the step S4 is input to an overall nested edge detection depth network to obtain an edge probability map, the overall nested edge detection depth network is trained by an open-source two-dimensional road surface data set and a corresponding crack label, and the data set input in the training process is also processed in the steps S3 and S4.
Further, in S6, the obtained edge probability map is converted into a gray scale map with values distributed in a range of 0 to 255, and the threshold Th1 — 3 × Ave is obtained in combination with the edge map mean Ave.
Further, in S7, based on the obtained and converted edge probability map and the threshold Th1, the weak edge lower than the threshold is removed, the strong edge not smaller than the threshold is retained, the selected connected domain of the strong edge is used as the image edge object, the position and direction of the image edge object are used as the input of the tensor voting algorithm, tensor coding is performed on the edge object region and the object direction, the tensor saliency field of the input edge object data is obtained, the more salient line segments in the obtained tensor saliency field are combined to construct a possible connection result, the connection result is combined with the strong edge map, and the edge map with the smaller connected domain is deleted by using the area parameter of the connected domain.
Further, in S8, based on the crack edge connection result acquired in S7 and the high-frequency component data acquired in S4, crack area and severity attribute information is acquired using the crack object region attribute feature.
Further, on the basis of the finally obtained position of the inference crack, correspondingly obtaining the position of the edge sub-object and the information of the connected domain of the edge sub-object; for two-dimensional data, determining the length and area attributes of the crack according to the characteristics of the connected domain of the crack edge object, and judging the severity of the crack by combining the gray level of high-frequency component data; for the three-dimensional data, determining the length and the area attribute of the crack according to the characteristics of the connected domain of the crack edge object, and determining the depth information of the crack by combining the high-frequency component data elevation map so as to determine the severity and the distribution attribute of the crack.
The scheme of the invention has the following beneficial effects:
the two-dimensional and three-dimensional pavement crack identification method based on the enhanced depth edge characteristics, provided by the invention, combines two-dimensional pavement and three-dimensional pavement data acquired by an actual application environment, and can be suitable for automatic crack identification tasks of various typical pavements such as asphalt pavements, cement pavements, coarse textures, fine textures, common texture pavements and the like acquired by a two-dimensional and three-dimensional data acquisition system; the method has the advantages that the frequency component characteristics of the cross section of the pavement and the multi-scale edge characteristics acquired by the deep learning network are fused, the edges of the pavement cracks are enhanced, on one hand, the flooding influence of a pavement data acquisition system on the characteristics of the crack edges due to uneven illumination (two-dimensional pavement data condition) and vehicle-carrying attitude fluctuation (three-dimensional pavement data condition) can be removed, on the other hand, the edge characteristics of different thicknesses and different depths of the cracks can be considered, and the accurate extraction of the two-dimensional and three-dimensional pavement data cracks is facilitated;
in the invention, the edge characteristic of the pavement crack after being decomposed and enhanced by the section component is enhanced, and a deep learning edge detection network is trained based on enhanced component data and a small amount of crack marking data, and the crack edge probability result of two-dimensional pavement enhanced data and three-dimensional pavement enhanced data can be simultaneously processed based on the deep learning edge detection network, so that a deep learning model is trained by utilizing the existing two-dimensional pavement marking data set, the defect that the network is difficult to train due to the insufficient pavement marking samples is overcome, accurate and comprehensive two-dimensional and three-dimensional pavement crack edge probability information is obtained under the limited marking condition, and a robust and steady crack detection mode can be provided for the actual highway pavement detection application;
other advantages of the present invention will be described in detail in the detailed description that follows.
Drawings
FIG. 1 is a flow chart of the steps of the present invention;
FIG. 2 is a schematic flow diagram of an integral nested edge detection depth network (HED) structure and an HED processing road surface edge enhancement data acquisition edge feature map according to the present invention;
FIG. 3 is a schematic diagram of tensor and edge object level tensor direction encoding results of the present invention, (a) voting generation; (b) a two-dimensional rod tensor saliency field; (c) HED strong edge object schematic; (c) a data tensor saliency field acquired in conjunction with tensor coding of edge object features; (d) HED strong edge join results based on tensor voting results;
FIG. 4 is a high-precision line-scanned three-dimensional pavement crack recognition example obtained by the present invention, (a) preprocessed three-dimensional pavement data; (b) obtaining an edge enhancement result of removing low-frequency components after frequency component decomposition; (c) the edge probability result is obtained by the HED deep network; (d) carrying out tensor voting and morphological post-processing on a crack identification result; (e) identifying the obtained crack attribute result based on the edge-enhanced pavement base map and the crack;
FIG. 5 is an example of two-dimensional pavement crack identification obtained by the present invention (a) pre-processed two-dimensional pavement data; (b) obtaining an edge enhancement result for removing uneven illumination after frequency component decomposition; (c) the edge probability result is obtained by the HED deep network; (d) carrying out tensor voting and morphological post-processing on a crack identification result; (e) and obtaining a crack attribute result based on the edge-enhanced pavement base map and the crack identification result.
Detailed Description
To make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In describing the present invention, for the sake of simplicity of explanation, the method or rules are depicted and described as a series of acts, which are not intended to be exhaustive or to limit the order of the acts. For example, the experimental procedures can be performed in various orders and/or simultaneously, and include other experimental procedures not described again. Moreover, not all illustrated steps may be required to implement a methodology or algorithm described herein. Those skilled in the art will recognize and appreciate that the methodologies and algorithms may be represented as a series of interrelated states via a state diagram or items.
As shown in fig. 1, embodiment 1 of the present invention provides a two-dimensional and three-dimensional pavement crack identification method based on an enhanced depth edge feature, which specifically includes the following steps:
and S1, reading in the three-dimensional road surface data/two-dimensional road surface data acquired by the road surface acquisition system. The two-dimensional road surface data refers to road surface optical data acquired by a vehicle-mounted two-dimensional camera. The three-dimensional pavement data refers to that a line scanning three-dimensional measuring sensor is utilized to collect a series of pavement section outlines along the measuring direction, and the series of pavement section outlines are spliced to obtain the three-dimensional data.
In the embodiment, two-dimensional road surface data is acquired by an optical camera and reflects the brightness information of the road surface; the three-dimensional road surface data are acquired by scanning the three-dimensional measuring sensor through a line, the relative elevation condition of the surface of the measured object is measured and obtained based on the triangulation principle, and the acquired three-dimensional data can reflect the elevation information of the measured road surface. The precision of the cross section of the two-dimensional road surface data is not less than 2mm, the precision of the cross section of the three-dimensional data is not less than 1mm, and the elevation resolution is higher than 0.5 mm.
And S2, preprocessing data. And correcting abnormal values (zero values and raised burrs) generated by system abnormality and environmental abnormality, and converting the section profile from an image space to an object space through a calibration file to obtain section data of the object space so as to correct system errors in the measurement system.
Due to the interference of the measuring environment (water and oil stains on the road surface or foreign matters in the measured area), a small amount of abnormal noise (zero point) may exist in the acquired data. In order to reduce the false detection of the zero-value points on the crack edges, the abnormal noise points are replaced by the non-abnormal sampling points in the cross section after the mean filtering, so that the preprocessed pavement section data are obtained. The actual vehicle-mounted road surface data acquisition system has system errors such as uneven sensor installation angle and light intensity distribution. These systematic errors weaken the characteristics of the pavement crack targets and therefore require correction of the data collected by the measurement sensors by means of calibration files.
S3: and removing the uneven illumination information of the three-dimensional road surface data attitude fluctuation/two-dimensional road surface data. In the process of collecting road surface data, due to the influences of bumping and fluctuation of driving, deformation and damage of the road surface and the like, the collected original three-dimensional data has obvious low-frequency amplitude attitude fluctuation information, and cracks are hidden in the macro change information. Due to the uneven lighting shadow of the pavement and the influence of pavement pits, deformation diseases and the like, the two-dimensional pavement also has obvious uneven brightness, and the crack edge information is easily interfered by the low-frequency uneven gray distribution.
Therefore, in order to reduce the influence of such low-frequency undulations in the road surface data on crack location and attribute information extraction, it is necessary to remove the low-frequency attitude undulation component contained in the data by using a correlation algorithm, for example, as described in patent CN108765376A, only the sum of the sparse component and the vibration component is retained as the high-frequency component. The processing can reduce the influence of the fluctuation of the travelling crane, uneven illumination and road surface deformation on crack detection on the premise of not losing crack information, so that the crack edge characteristic enhancement and the attribute information extraction are facilitated.
Specifically, for the three-dimensional road surface data obtained in S2, performing component decomposition on each cross section, removing low-frequency component influence caused by attitude fluctuation, road surface smooth deformation and the like in the dynamic vehicle-carrying acquisition process, and enhancing the high-frequency edge characteristics of cracks; for the two-dimensional road surface data acquired in S2, each cross section is decomposed according to frequency components, and the influence of low-frequency components due to uneven illumination shading of the road surface and the like is removed, thereby enhancing the high-frequency edge characteristics of the crack.
And S4, acquiring edge enhancement data. And analyzing the components of the road surface data on the basis of the cross section data of the road surface, splicing the high-frequency components of all the cross sections to form high-frequency component data with enhanced edge characteristics, and taking the high-frequency component data as the input of subsequent processing.
In order to enable the acquired edge enhancement data to be better connected with a deep learning network, the data after edge enhancement is uniformly converted into a gray scale map with the numerical value distribution in the range of 0-255.
And S5, performing integral nested edge detection deep network training, and acquiring a test data edge probability graph according to the training model. And inputting the data obtained in the step S4 into the integral nested edge detection deep network to obtain an edge probability map. The depth network is trained by an open-source two-dimensional pavement data set and corresponding crack labels, and the data set input in the training process is also processed in steps S3 and S4 to enhance the crack edge characteristics of the input data.
Wherein, the open source road surface data set who participates in the pre-training contains corresponding mark, includes: the system comprises a two-dimensional asphalt pavement data set FISSURESDataset, a two-dimensional asphalt pavement data set deep crack Dataset, an ESAR two-dimensional pavement data set and an LCMS three-dimensional pavement data set.
The overall nested edge detection deep network (HED) structure is shown in fig. 2, and is improved based on a 16-layer visual set group network (VGG), an output layer is connected with a convolutional layer, the last pooling layer of the VGG and all the following connection layers are removed, and the overall image is trained and predicted, so that the feeling of a feature map can be increased, and the problem of low resolution of the feature map can be alleviated. Meanwhile, the HED belongs to a multi-level and multi-scale network and has robustness to noise. The HED respectively connects a side output layer to the last convolution layer of each sub-module (stage), the edge characteristics contained in different output layers are different, the side output image is subjected to deconvolution processing, the sizes of the side output layers are consistent with those of the input image, and finally the multi-scale edge characteristics are fused by utilizing the characteristic fusion layer to form a fusion edge output layer. For real road surface data, edge pixels and non-edge pixels are severely unbalanced, and the HED uses a class balance weight strategy to automatically balance the loss between edge/non-edge classes.
The HED can acquire the road surface data edge feature map output in S4, and due to the difference in the thickness and depth of the crack itself and the interference of the road surface texture background, the edge feature map contains not only edges of different degrees, but also edge information of other objects, and further processing is required.
And S6, further processing the edge probability map to obtain strong edge information of the road surface data. The fusion layer result obtained in S5 is used as an edge probability map of the input road surface data. Due to the phenomena of weak texture of the background of the road surface, edge influence and uneven intensity of the edge caused by different depths of the cracks, it is necessary to further judge and process the edge probability map output by the depth network in combination with the continuity of the cracks. And converting the edge probability map output by the depth network into a gray scale map with the value distribution in the range of 0-255, and combining the edge map average value Ave to obtain the threshold Th 1-3 Ave.
S7: strong edge selection and connection. Based on the edge probability map acquired at S6 and the threshold Th1, weak edges below the threshold are removed, and strong edges not less than the threshold are retained. And (3) taking the selected connected domain with the strong edge as an image edge object, taking the position and the direction of the image edge object as the input of a tensor voting algorithm, and combining tensor coding of the edge object area and the object direction to obtain a tensor significance field of the input edge object data. And combining more significant line segments in the acquired tensor significance field to construct a possible connection result. And combining the connection result with the strong edge graph, and deleting the edge graph with a smaller connected domain by using the area parameter of the connected domain.
Fig. 3(a), (b) show the Tensor Voting (TV) algorithm generation and 2D wand tensor saliency fields. The TV algorithm infers salient structures from the data through tensor representations of features and nonlinear voting. The TV algorithm generally uses two-dimensional points as input, and the present embodiment improves the TV algorithm by using image edge object sub-blocks with direction and position features as input. Specifically, a second-order symmetric non-negative tensor field T is formed by using the position and direction features of the image sub-object, wherein T is (lambda) 1 -λ 2 )e 1 e 1 T+λ 2 (e 1 e 1 T+e 2 e 2 T). By decomposing the tensor field, meaningful curve information can be obtained. Lambda 1 And λ 2 Is a characteristic value in descending order, e 1 And e 2 Are the corresponding feature vectors. (lambda 1 -λ 2 ) Expressing the significance of the rod tensor, it shows the significance of the rod as an eigenvector lambda 1 As the basic curve element of the curve normal. From the curve structure saliency values of each image object element, a probability map representing the curve structure occurring at a certain position, called tensor saliency field of the sub-objects, can be obtained, as shown in fig. 3 (d).
The direction of the pixel points in each sub-object is obtained from the direction of the whole sub-object, and the improved object-level direction tensor voting can be combined with the proximity characteristic and the direction characteristic of the crack to properly extend the discontinuous edge characteristics. As shown in fig. 3(c), the selected sub-object is displayed, and the position of the sub-object and its orientation are used as input for the TV. Therefore, combining the sub-object region with the direction tensor coding of the object can obtain the result after strong edge connection, as shown in fig. 3 (e). And deleting the edge map with the smaller connected domain by using the connected domain area parameter, wherein the connected domain area threshold Th2 set at the position can be set according to requirements, and the Th2 threshold is default to 100 for high-precision two-dimensional and three-dimensional data with the resolution precision of 1-2mm in the transverse sampling of the data.
S8: and extracting crack attribute information of the two-dimensional/three-dimensional road surface. And acquiring attribute information such as crack area, severity and the like by using the attribute features of the crack object region based on the crack edge connection result acquired at S7 and the high-frequency component data acquired at S4. Specifically, on the basis of the finally obtained position of the inference crack, the position of the edge sub-object and the connected domain information thereof are correspondingly obtained. For two-dimensional data, the length, the area attribute and the like of the crack can be determined according to the characteristics of the connected domain of the crack edge object, and the severity of the crack can be judged by combining the gray level of the high-frequency component data. For three-dimensional data, the length, the area attribute and the like of the crack can be determined according to the characteristics of the connected domain of the crack edge object, the depth information of the crack can be judged by combining a high-frequency component data high-level graph, and the severity and the distribution attribute of the crack are further judged. The two-dimensional/three-dimensional pavement crack attribute information can provide more comprehensive results for the severity and distribution of pavement cracks.
The effectiveness and accuracy of the present embodiment will be further described below with reference to the line-scanned three-dimensional road surface data acquired from the actual road surface and the open-source two-dimensional road surface data set.
1. And (6) testing data. In order to verify the universality of the method, on the basis of HED (high efficiency empirical mode) trained by the FISSURES Dataset, the deep crack Dataset, the ESAR two-dimensional Dataset and the LCMS three-dimensional pavement Dataset, an Aigle-RN two-dimensional pavement Dataset and a three-dimensional pavement Dataset (the transverse resolution is 1mm, and the elevation resolution is less than or equal to 0.5mm) acquired by equipment in CN112270677A are selected as test data
2. And (3) performing a three-dimensional pavement data crack detection test. Fig. 4(a) illustrates a high-precision line-scan three-dimensional road surface affected by carrier attitude fluctuation and road surface deformation damage, the edge enhancement result obtained by removing low-frequency components after frequency component decomposition is shown in fig. 4(b), the edge probability result obtained by the HED is shown in fig. 4(c), the crack recognition result obtained by tensor voting and morphological post-processing is shown in fig. 4(d), and the crack attribute information obtained based on the edge enhancement road surface map and the crack recognition result is shown in fig. 4 (e).
3. And (5) performing a two-dimensional pavement crack recognition experiment. Fig. 5(a) illustrates two-dimensional road surface data affected by uneven lighting in one of the Aigle-RN two-dimensional road surface data sets, an edge enhancement result obtained by decomposing frequency components and removing uneven lighting is shown in fig. 5(b), an edge probability result obtained by the HED depth network is shown in fig. 5(c), a crack identification result obtained by tensor voting and morphological post-processing is shown in fig. 5(d), and crack attribute information obtained based on an edge enhancement road surface base map and the crack identification result is shown in fig. 5 (e).
According to the actual operation results, the influence of environment and vehicle loading in actual pavement data can be effectively removed by fully utilizing edge enhancement processing and deep network edge characteristics, and the crack identification method which is accurate, effective and universal is carried out by means of dynamically acquired two-dimensional and three-dimensional pavement data.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge features is characterized by comprising the following steps:
s1, reading in three-dimensional pavement data/two-dimensional pavement data acquired by a pavement acquisition system;
s2, preprocessing data;
s3, removing uneven illumination information of the three-dimensional road surface data attitude fluctuation/two-dimensional road surface data;
s4, acquiring edge enhancement data;
s5, performing integral nested edge detection deep network training, and acquiring a test data edge probability graph according to a training model;
s6, converting the edge probability map to obtain strong edge information of the road surface data;
s7, selecting and connecting strong edges;
and S8, extracting crack attribute information of the two-dimensional/three-dimensional road surface.
2. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 1, wherein the two-dimensional pavement data is pavement optical data obtained by using a vehicle-mounted two-dimensional camera, and the three-dimensional pavement data is obtained by using a line scanning three-dimensional measuring sensor to collect a series of pavement section profiles along a measuring direction and splicing the series of pavement section profiles.
3. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 2, wherein abnormal values generated by system abnormality and environmental abnormality are corrected in S2, the cross section profile is converted from image space to object space through a calibration file, the cross section data of the object space is obtained, and the system error in the measurement system is corrected.
4. The method for identifying two-dimensional and three-dimensional pavement cracks based on enhanced depth edge features of claim 3, wherein in step S3, for the obtained three-dimensional pavement data/two-dimensional pavement data, each cross section is subjected to component decomposition to remove the influence of low-frequency components and enhance the high-frequency edge features of cracks.
5. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 4, wherein in S4, the high-frequency components of the cross sections are spliced to form the high-frequency component data with enhanced edge characteristics, and the data with enhanced edge characteristics are uniformly converted into a gray scale map with the numerical value distribution in the range of 0-255.
6. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 5, wherein the data obtained in S4 is input into an overall nested edge detection depth network in S5 to obtain an edge probability map, the overall nested edge detection depth network is trained by open-source two-dimensional pavement data sets and corresponding crack labels, and the data sets input in the training process are also processed in S3 and S4.
7. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 6, wherein the obtained edge probability map is converted into a gray scale map with the numerical value distribution in the range of 0-255 in S6, and the threshold Th1 is obtained by combining the edge map mean Ave.
8. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 7, wherein in S7, based on the obtained and converted edge probability map and the threshold Th1, weak edges lower than the threshold are removed, strong edges not smaller than the threshold are retained, a connected domain of the selected strong edges is used as an image edge object, the position and direction of the image edge object are used as input of a tensor voting algorithm, a tensor saliency field of the input edge object data is obtained by combining with tensor coding of an edge object region and an object direction, a possible connection result is constructed by combining with more significant line segments in the obtained tensor saliency field, the connection result is combined with the strong edge map, and the edge map with a smaller connected domain is deleted by using an area parameter of the connected domain.
9. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 8, wherein crack object region attribute features are used to obtain crack area and severity attribute information based on the crack edge connection result obtained at S7 and the high frequency component data obtained at S4 in S8.
10. The two-dimensional and three-dimensional pavement crack recognition method based on the enhanced depth edge feature of claim 9, characterized in that the edge sub-object position and its connected domain information are correspondingly obtained on the basis of the finally obtained inference crack position; for two-dimensional data, determining the length and area attributes of the crack according to the characteristics of the connected domain of the crack edge object, and judging the severity of the crack by combining the gray scale of high-frequency component data; for the three-dimensional data, determining the length and the area attribute of the crack according to the characteristics of the connected domain of the crack edge object, and determining the depth information of the crack by combining the high-frequency component data elevation map so as to determine the severity and the distribution attribute of the crack.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210601919.0A CN114862892A (en) | 2022-05-30 | 2022-05-30 | Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210601919.0A CN114862892A (en) | 2022-05-30 | 2022-05-30 | Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114862892A true CN114862892A (en) | 2022-08-05 |
Family
ID=82640978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210601919.0A Pending CN114862892A (en) | 2022-05-30 | 2022-05-30 | Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114862892A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665215A (en) * | 2023-05-25 | 2023-08-29 | 北京航星永志软件技术有限公司 | Image salient region extraction method, device, computer equipment and storage medium |
-
2022
- 2022-05-30 CN CN202210601919.0A patent/CN114862892A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665215A (en) * | 2023-05-25 | 2023-08-29 | 北京航星永志软件技术有限公司 | Image salient region extraction method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Automatic pavement defect detection using 3D laser profiling technology | |
| Akagic et al. | Pavement crack detection using Otsu thresholding for image segmentation | |
| Kim et al. | Comparative analysis of image binarization methods for crack identification in concrete structures | |
| Min et al. | Real time detection system for rail surface defects based on machine vision | |
| US11551341B2 (en) | Method and device for automatically drawing structural cracks and precisely measuring widths thereof | |
| CN109087274B (en) | Electronic device defect detection method and device based on multi-dimensional fusion and semantic segmentation | |
| Ouma et al. | Wavelet-morphology based detection of incipient linear cracks in asphalt pavements from RGB camera imagery and classification using circular Radon transform | |
| CN107154040B (en) | Tunnel lining surface image crack detection method | |
| Wang et al. | Comparison analysis on present image-based crack detection methods in concrete structures | |
| CN110473187B (en) | Object-oriented line scanning three-dimensional pavement crack extraction method | |
| Tran et al. | A two-step sequential automated crack detection and severity classification process for asphalt pavements | |
| Li et al. | Automatic segmentation and enhancement of pavement cracks based on 3D pavement images | |
| CN107798293A (en) | A kind of crack on road detection means | |
| Garbowski et al. | Semi-automatic inspection tool of pavement condition from three-dimensional profile scans | |
| CN111489352A (en) | Tunnel gap detection and measurement method and device based on digital image processing | |
| Zawad et al. | A comparative review of image processing based crack detection techniques on civil engineering structures | |
| CN114705689A (en) | Unmanned aerial vehicle-based method and system for detecting cracks of outer vertical face of building | |
| Mathavan et al. | Detection of pavement cracks using tiled fuzzy Hough transform | |
| Vivekananthan et al. | Concrete bridge crack detection by image processing technique by using the improved OTSU method | |
| CN111882664A (en) | Multi-window accumulated difference crack extraction method | |
| CN114862892A (en) | Two-dimensional and three-dimensional pavement crack identification method based on enhanced depth edge characteristics | |
| Gui et al. | Object-based crack detection and attribute extraction from laser-scanning 3D profile data | |
| WO2013173464A1 (en) | Apparatus and method for tire sidewall crack analysis | |
| Xie et al. | A four-stage product appearance defect detection method with small samples | |
| CN104598906A (en) | External-profile detection method and device for vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |