CN110197157B - Pavement crack growth detection method based on historical crack data - Google Patents

Pavement crack growth detection method based on historical crack data Download PDF

Info

Publication number
CN110197157B
CN110197157B CN201910469277.1A CN201910469277A CN110197157B CN 110197157 B CN110197157 B CN 110197157B CN 201910469277 A CN201910469277 A CN 201910469277A CN 110197157 B CN110197157 B CN 110197157B
Authority
CN
China
Prior art keywords
crack
image
historical
data
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910469277.1A
Other languages
Chinese (zh)
Other versions
CN110197157A (en
Inventor
胡钊政
陈佳良
王相龙
张帆
陶倩文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University of Technology WUT
Original Assignee
Wuhan University of Technology WUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University of Technology WUT filed Critical Wuhan University of Technology WUT
Priority to CN201910469277.1A priority Critical patent/CN110197157B/en
Publication of CN110197157A publication Critical patent/CN110197157A/en
Application granted granted Critical
Publication of CN110197157B publication Critical patent/CN110197157B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/80—Recognising image objects characterised by unique random patterns
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A30/00—Adapting or protecting infrastructure or their operation
    • Y02A30/60—Planning or developing urban green infrastructure

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a road surface crack growth detection method based on historical crack data, which comprises the following steps: GPS initial positioning, namely extracting a plurality of similar images close to the current road surface image by comparing the current positioning information with position information in historical map data; obtaining precisely matched image data in a plurality of similar images close to the current road surface image after the image-level positioning is matched with the characteristic points; the pixel level positions image characteristic points matched through the ORB, calculates an H matrix, and maps historical mark crack pixels to a current crack image by using the H matrix; historical crack pixel distribution mapped in the current crack image is analyzed based on the historical cracks of the RGM, the intensity distribution of the pixels of the mapped cracks is represented by a Gaussian model, and finally the pixel values meeting the conditions are divided into cracks. The method of the invention provides an effective and reliable strategy for researching the change of the crack state along with time by referring to historical crack data, thereby greatly simplifying and improving the detection and identification of the crack.

Description

Pavement crack growth detection method based on historical crack data
Technical Field
The invention relates to an image processing technology, in particular to a road surface crack growth detection method based on historical crack data.
Background
At present, some achievements have been made in the aspect of pavement crack detection, for example, in the applied patent CN106548182, 2016, 11, month and 02, and the patent name "pavement crack detection method and device based on deep learning and principal cause analysis" discloses a pavement crack detection method based on a convolutional neural network and principal cause analysis. The patent CN106324084, 2016, 8, 30, and the patent name "crack depth detection method" discloses a method and an apparatus for detecting the depth value of a crack.
In recent years, with the development of pavement crack detection technology being accelerated, the requirement of a pavement crack detection method, particularly for crack propagation analysis on a pavement, is increasingly urgent, a current machine learning-based supervision method is a main means for pavement crack detection, and the method is applied on the premise that a large amount of marking data are trained and is difficult to be widely applied in practice. The patent provides a method for utilizing historical image data of a certain time interval to carry out mapping comparison with an existing crack image, and training data amount is reduced while detection precision is improved.
Disclosure of Invention
The invention aims to solve the technical problem of providing a pavement crack growth detection method based on historical crack data aiming at the defects in the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows: a pavement crack growth detection method based on historical crack data comprises the following steps:
1) Acquiring a current pavement crack image in real time, and synchronously acquiring positioning information corresponding to the current pavement crack image;
2) Initial positioning: according to the current positioning information and the position information in the historical map data, extracting historical image data within a threshold distance from the current positioning information from the historical road surface image data, and selecting a plurality of historical images similar to the current road surface image from the historical road surface image data;
3) Image level localization: finding a closest road surface image from the roughly matched similar images in the step 2), wherein the method specifically comprises the following steps:
3.1 Gray-scale processing the image;
3.2 Calculating the Hamming distances of all feature descriptors of the current gray image and the similar image, comparing the Hamming distances to realize coarse matching of feature points, and then finding out the same feature points of the current gray image and the similar image, wherein the matched feature points are used as pavement fingerprint information;
3.3 By comparing the number of the same feature points, a closest road surface image is found in the rough matching similar images;
4) Pixel level positioning: mapping the historical cracks which are closest to the marks in the pavement image into the current pavement crack image;
5) And detecting all pixels belonging to newly grown cracks in the acquired crack image based on the RGM (Region Growth Method) mapping crack analysis of the current pavement crack image.
According to the scheme, the method for acquiring the feature descriptors in the step 3.2) comprises the following steps:
3.2.1 Detect image feature points: detecting angular points as image characteristic points through a Harris algorithm;
3.2.2 Add direction information: adding direction information to the extracted characteristic points to enable the directions of the extracted characteristic points to be unchanged, wherein the directions are obtained by calculating entropy values of pixel blocks through moments of images in a circular window;
3.2.3 ORB feature point matching:
selecting n pairs of features on the feature points to form a mapping matrix s, wherein the size of the matrix s is 2X 2n, the elements of the matrix s are the coordinates of each feature pair on the X axis and the Y axis, then obtaining an affine transformation matrix R by utilizing the directions from the feature points to the centroid, and obtaining a new description matrix s by utilizing the matrix R for calculation θ And combining the BRIEF descriptor to obtain the ORB feature descriptor.
According to the scheme, the method for judging the image feature points in the step 3.2.1) comprises the following steps: comparing the difference values of pixel points on a circular window formed by one pixel point P in the image and a plurality of surrounding points, wherein the sum of the difference values of the plurality of points is N, and when N is greater than a judgment standard, judging the point as an image feature point;
Figure BDA0002080340020000031
Figure BDA0002080340020000041
wherein, I (x) is the gray value of the current pixel point, I (P) is the gray value of the pixel point P, epsilon is a set threshold, circle (P) is the radiation range on the circular window around the pixel point P, and the formula substitutes the pixel points meeting the range requirement.
According to the scheme, the pixel-level positioning in the step 4) is to map the marked historical crack pixels into the current crack image.
According to the scheme, the pixel-level positioning in the step 4) is to calculate the H matrix of the two images through the image feature points matched by the ORB, and then map the marked historical crack pixels into the current crack image by using the H matrix.
According to the scheme, the specific steps of the multi-scale positioning-based crack mapping in the step 4) are as follows:
assuming that the road is a plane, under the pinhole camera model, the basic geometry can be described by using a homography matrix, and two linear constraints on the homography matrix are known, and the historical crack label is mapped to the query image through the following relationship:
Figure BDA0002080340020000042
wherein n is the number of historical crack label pixels, [ u' i y′ i ] T Coordinates for mapping fracture data in historical data, [ u ] u i y i ] T The coordinates of the historical crack label pixels.
According to the scheme, the historical fracture analysis based on RGM in the step 5) comprises the following specific steps:
5.1 Map crack gray value distribution analysis: taking the query crack labels mapped in the above steps as 'ideal' initial seed points, and starting from the seed points, expanding the region by searching for adjacent points with similar attributes to the seed points, wherein the attributes comprise color and intensity; calculating pixel values from the mapping corresponding relation on the query image through the mapping labels, and drawing an image intensity distribution histogram of the mapping crack data by utilizing a statistical principle;
5.2 The intensity distribution of the mapped crack pixels is represented by a gaussian model:
establishing a Gaussian model to represent the distribution, using the distribution characteristics as operators, and calculating the corresponding mean value and standard deviation according to the following formula:
Figure BDA0002080340020000051
Figure BDA0002080340020000052
and N is the number of the pixels for mapping the crack labels.
5.3 Growth crack analysis: when the intensity of a certain point in the image satisfies the following condition, the point is divided into cracks:
I(p μ ,p v )∈[0,ω+λσ]
in the formula, λ is a constant value determined according to the actual application.
According to the scheme, the positioning information in the step 1) is GPS positioning information.
The invention has the following beneficial effects: carrying out crack detection from three steps of GPS primary positioning, image-level positioning and historical crack analysis based on RGM; the GPS initial positioning extracts a plurality of similar images close to the current road surface image by comparing the current positioning information with the position information in the historical map data. And detecting angular points through a Harris algorithm in image-level positioning, adding direction information to the extracted characteristic points to enable the directions of the extracted characteristic points to be unchanged, and performing ORB characteristic point matching to obtain precisely matched image data. And mapping the matched historical crack label to a query image based on the historical crack analysis of the RGM, representing the intensity distribution of the pixels of the mapped crack by using a Gaussian model, and finally dividing the pixel values meeting the conditions into cracks. According to the method, historical crack data are referred, an effective and reliable strategy for researching the change of the crack state along with time is provided, and the detection and identification of the crack are greatly simplified and improved.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a flow chart of a method of an embodiment of the present invention;
FIG. 2 is a flow diagram of a method of an embodiment of the invention;
FIG. 3 is a schematic diagram of coordinate matching initial positioning according to an embodiment of the present invention;
FIG. 4 is a schematic view of a seed point and a growth area of an embodiment of the present invention;
FIG. 5 is an image intensity histogram of mapped fracture data according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
As shown in fig. 1 and 2, a crack growth detection method based on historical crack data according to an embodiment of the present invention includes: multi-scale positioning; multi-scale localization based crack mapping and RGM based mapped crack analysis. The multi-scale positioning comprises GPS initial positioning, image-level positioning and pixel-level positioning; RGM-based mapping crack analysis includes historical crack gray value distribution analysis, gaussian model construction and growth crack analysis.
The GPS initial positioning method comprises the following specific steps:
s1, acquiring a current pavement crack image in real time through an image sensing module arranged on a vehicle to be positioned, and synchronously acquiring current positioning information through a positioning module;
and S2, extracting a plurality of similar images close to the current road surface image from the historical map data by comparing the current positioning information with the position information in the historical map data, and finally acquiring the historical image data within a threshold distance. The following formula is satisfied:
d ji ＝dist(G j ,G i )
Pos＝{G i |d ij ≤k}
wherein G is j GPS coordinates, G, for jth query for fracture data i Is the GPS coordinate of the ith historical fracture data, d is the GPS distance between the current fracture data and the historical fracture data, k is the threshold distance of the candidate historical fracture data, and Pos is a set of historical fracture candidate images.
Meanwhile, the positioning accuracy of the GPS in practical application is 10 meters, and in the step, the positioning accuracy is used as a threshold value to primarily screen historical cracks, so that the initial positioning of the GPS is completed and is used as a first step of multi-scale positioning.
The image level positioning comprises the following specific steps:
s1, detecting image feature points: detecting angular points through a Harris algorithm, namely comparing the difference value of pixel points on a circle consisting of a pixel point P and a plurality of surrounding points in an image, wherein the formula is as follows:
Figure BDA0002080340020000081
Figure BDA0002080340020000082
wherein, I (x) is the gray value of the current pixel point, I (P) is the gray value of the pixel point P, epsilon is a set threshold, the sum of the response function values of a plurality of points is N, and when N is greater than the judgment standard, the point is an image feature point;
generally, an integer part of 3/4 of the value N is taken as a judgment standard;
s2, adding direction information: and adding direction information to the extracted feature points to enable the directions of the extracted feature points to be unchanged. The orientation is calculated from the moments of the image within the circular window. The moments of the image are defined as follows:
Figure BDA0002080340020000083
thus, the entropy of the image is calculated from each time instant as follows:
Figure BDA0002080340020000091
the orientation of the pixels is defined as follows:
θ＝arctan(m 10 ,m 01 )
wherein m is pq The moment of the image block is represented by (p, q epsilon (0, 1)), I (x, y) represents a gray value at the coordinate (x, y), and theta is an included angle between the centroid and the characteristic point;
s3, matching ORB characteristic points:
selecting n pairs of features on the feature points to form a matrix s:
Figure BDA0002080340020000092
the size of the matrix s is 2 x 2n, where x i ,y i (i belongs to (1, n)) represents the coordinates of the ith feature pair in X and Y axes, an affine transformation matrix R is obtained by utilizing the directions from feature points to the centroid, and a new description matrix s is obtained by utilizing the matrix R for calculation θ The formula is as follows:
Figure BDA0002080340020000093
combining with BRIEF descriptor to obtain ORB feature descriptor:
g n (P,θ)＝f n (P)|(x i ,y i )∈S θ
meanwhile, the selected n value is 256, according to the obtained 256-dimensional ORB feature descriptors, the Hamming distances of all feature descriptors of the current gray image to be positioned and the similar images are calculated, the Hamming distances are compared to realize coarse matching of feature points, wrong matching is removed by using a RANSAC algorithm, and therefore the same feature points of the current gray image to be positioned and the similar images are found, and the matched feature points are used as road surface fingerprint information; and by comparing the number of the same characteristic points, finding out a closest road surface image from the roughly matched similar images, wherein the number of the characteristic points matched with the image to be positioned is the largest, thereby realizing the precise matching of the image to be positioned by utilizing the fingerprint information.
The pixel-level positioning is used for solving an H matrix of two images, and then mapping the marked historical crack into the current crack image by using the H matrix, and the specific steps are as follows:
assuming that the road is a plane, under the pinhole camera model, the basic geometry can be described by a homography matrix, and the specific formula is as follows:
Figure BDA0002080340020000101
Figure BDA0002080340020000102
h i representing the components of the matrix H.
The above formula is rewritten as follows:
Figure BDA0002080340020000103
Figure BDA0002080340020000104
two linear constraints on the homography matrix can be derived from the above equation:
Figure BDA0002080340020000111
mapping the historical crack labels onto the query image by using the computed homography matrix according to the following relation:
Figure BDA0002080340020000112
wherein n is a historical crack label pixel, [ u' i y′ i ] T For the coordinates of the mapped fracture data in the historical data, [ u ] i y i ] T The coordinates of the historical crack label pixels.
Thus, the mapped fracture label on the query image may be represented by a set of two-dimensional image coordinates, as follows:
Q＝{[u′ i y′ i ] T }(i＝1,2,…n)
the specific steps of the historical fracture analysis based on RGM are as follows:
s1, analyzing the distribution of historical crack gray values: the query crack labels mapped in the above steps are taken as "ideal" initial seed points from which the region is expanded by finding neighboring points with similar attributes (e.g., similar color, intensity, etc.) to the seed points. By mapping the labels, pixel values can be calculated from the mapping correspondences on the query image. And drawing an image intensity distribution histogram of the mapping crack data by using a statistical principle.
S2, representing and mapping the intensity distribution of the crack pixels by a Gaussian model:
all of the above-mentioned pixel values I (μ ', ν') associated with the mapped crack signatures follow a certain pattern, i.e. gaussian distribution, and thus can be used as a constraint for crack propagation. To this end, a gaussian model is built to represent the distribution and its distribution characteristics are used as operators. The corresponding mean and standard deviation can be calculated as:
Figure BDA0002080340020000121
Figure BDA0002080340020000122
and N is the number of the pixels of the mapping crack label.
It should be noted that each query fracture image has a unique developed gaussian model, and therefore the image used is robust and reliable in performing region growing calculations. Through the established Gaussian model, whether the adjacent images have similar properties with the seed points or not can be quickly determined. Meanwhile, when performing the region growing calculation, since the crack region in the image generally has a low image intensity, the range of the image intensity, such as the average value and the standard deviation, may be set from the gaussian model parameter.
S3, growth crack analysis: when the intensity of a certain point in the image satisfies the following condition, the point is divided into cracks:
I(p μ ,p v )∈[0,ω+λσ]
in the formula, λ is a constant value determined according to practical application.
In another embodiment of the invention:
the reference-based fracture analysis method of the embodiment of the invention comprises three main modules, namely 1) multi-scale positioning; 2) Mapping the historical crack image to a query crack image; 3) And (5) performing crack post-treatment and analysis. Each of the queried fracture data and historical fracture data contains GPS information and a fracture image. Furthermore, the historical crack labels in the historical crack data, whether extracted manually or automatically, are represented as a set of pixels (pixel level) belonging to cracks in the road surface image. Thus, each historical fracture data is represented using a set of points as follows:
m i ＝{G i ,I i ,L i } i∈(1,n)
wherein n is the number of historical fracture data. G i As GPS information, I i As an image of a road surface crack, L i Is a historical crack label.
The specific flow is shown in figure 1.
The multi-scale positioning module is mainly used for establishing image corresponding relation between current fracture data and historical fracture data. The method adopts a strategy from coarse to fine, and realizes multi-scale positioning by three steps of GPS initial positioning, image-level positioning and pixel-level positioning.
Primary positioning by a GPS (global positioning system):
the GPS data is initially positioned, and the process is shown in FIG. 3. Let G j GPS coordinates, G, for jth query for fracture data i GPS coordinates for the ith historical fracture data. The distance between the two can be calculated as:
d ji ＝dist(G j ,G i )
and matching the GPS data to obtain a group of historical crack candidate images, wherein the related GPS coordinates and the inquiry GPS coordinates of the historical crack candidate images are within a threshold distance. GPS-based primary positioning allows a limited number of candidate images to be extracted from a large number of historical fracture images collected. The mathematical description of this task is as follows:
Pos＝{G i |d ij ≤k}
where k is a threshold distance for selecting candidate historical fracture data. It determines whether the ith historical fracture data is sufficiently close to the jth query fracture data. In practical applications, the threshold distance is 10 meters depending on the error of the GPS positioning.
And starting from the initial positioning result, image matching is applied to realize image-level positioning. The purpose of image-level localization is to find the "most recent" historical fracture image among the candidate historical fracture data. "nearest" means that the distance between the camera position associated with the query fracture image and the matching historical fracture image is the smallest of all historical fracture images. The method utilizes matching of pairs of local image feature points to achieve image-level localization.
And matching the local image feature point pairs by utilizing an ORB algorithm. ORB is an image matching algorithm that combines oFAST (FAST with directions) and rBRIEF (rotating BRIEF). All images are represented by some local feature points. The local feature point pairs of the two images represent the same region in the two images. Wherein oFAST is used for feature point detection, and rBRIEF is used for feature descriptor calculation.
oFAST first uses the Harris angular measure to select different feature points. Secondly, adding direction information to make the extracted characteristic points be direction-invariant. The orientation is calculated from the moments of the image within the circular window. The moments of the images are defined as follows:
Figure BDA0002080340020000141
therefore, the entropy of the image block is calculated from each time instant as follows:
Figure BDA0002080340020000142
the direction of the image block is defined as follows:
θ＝atan2(m 01 ,m 10 )
where atan2 is a variation of the tangent function tan and the BRIEF descriptor is a bit string description of an image patch constructed from a set of binary intensity tests. Consider a smooth image pixel P, two arbitrarily positioned binary test indices x and y, and compare the logical results of their image intensities:
Figure BDA0002080340020000151
where P (x) is the intensity of the image at a certain point x. Thus, the BRIEF descriptor is defined as a vector of n binary test indices x and y:
Figure BDA0002080340020000152
in the literature, there are many solutions on how to choose n binary tests. In an embodiment of the invention a gaussian distribution around the center of the image block is used and the vector length n =256 is chosen. Thus, the ORB property descriptor is represented by a 256-bit string. In order to keep the BRIEF descriptors from changing with rotation, the BRIEF description is made according to the direction of the key points. For binary testing of any property set n, the following 2 × n matrix is defined
Figure BDA0002080340020000153
From the direction θ of the image block, the corresponding rotation matrix can be calculated as follows:
Figure BDA0002080340020000161
then a controllable S can be constructed by rotating the matrix θ As follows:
S θ ＝R θ S
the rotation invariant descriptor, also called ORB descriptor, can thus be computed as follows:
g n (Ρ,θ)＝f n (Ρ)|(x i ,y i )∈S θ
mapping historical fracture labels to query images:
once the "recent" historical fracture images are obtained, the potential geometric relationships between the queried and historical images can be obtained. Assuming that the sidewalk is a plane, under the pinhole camera model, the bottom layer geometry can be described by a homography matrix:
Figure BDA0002080340020000162
Figure BDA0002080340020000163
h i representing the components of the matrix H.
The above two formulas can be rewritten as follows:
Figure BDA0002080340020000164
Figure BDA0002080340020000165
two linear constraints on the homography matrix can be derived from the above equation:
Figure BDA0002080340020000171
since the homography matrix can be determined in scale, the homography matrix can be calculated from at least 4 point pairs. In practical applications, a Direct Linear Transformation (DLT) can be applied to calculate the homography matrix and the results optimized with the Levenberg-Marquardt (LM) (Levenberg-Marquardt method).
Using the computed homography matrix, historical fracture labels may be mapped onto the query image as shown in the following equation:
Figure BDA0002080340020000172
where n is the historical crack label pixel. [ u' i y′ i ] T Coordinates for mapping fracture data in the historical data. [ u ] of i y i ] T The coordinates of the historical crack label pixels.
Thus, the mapped fracture label on the query image may be represented by a set of two-dimensional image coordinates, as shown in the following equation:
Q＝{[u′ i y′ i ] T }(i＝1,2,…n)
crack post-treatment and zone growth analysis:
in practical application, the query crack labels can be quickly realized by mapping the historical crack labels into the query crack images. Since the crack condition may be exacerbated in the time interval between the query and the historical crack data acquisition, there are still some pixels that belong to the newly created crack that need to be detected. Since RGM relies on good fracture seed points, the query fracture label mapped in the above step can be used as an "ideal" initial seed point. Starting from these seed points, the region is expanded by finding neighboring points with similar attributes (e.g., similar color, intensity, etc.) as the seed points, as shown in fig. 4. The red dot in the middle is the initial seed point and the blue dot is the growth area.
By mapping the labels, pixel values can be calculated from the mapping correspondences on the query image. All of these pixel values associated with the mapped crack labels follow a pattern, as shown in FIG. 5. FIG. 5 illustrates an exemplary image intensity distribution of a pixel mapping crack similar to a Gaussian model that may be used as a constraint on crack propagation. Thus, embodiments of the present invention develop a Gaussian model to accomplish this task. The corresponding mean and standard deviation can be calculated as:
Figure BDA0002080340020000181
Figure BDA0002080340020000182
and N is the number of the pixels of the mapping crack label.
It should be noted that each query fracture image has a unique developed gaussian model and is therefore robust and reliable for region growing. From the calculated gaussian model, it can be quickly determined whether the neighboring images have similar properties to the seed points. Since crack regions in an image typically have lower image intensities, the range of image intensities can be set from gaussian model parameters (such as mean and standard deviation). Therefore, a certain point is classified as a crack when its strength satisfies the following condition:
I(p μ ,p v )∈[0,ω+λσ]
in the formula, λ is a definite ratio and is empirically set in practical application.
Therefore, the RGM can apply the above equation to detect all neighboring image points near the seed point. With RGM, all pixels belonging to the newly grown crack in the query crack image can be detected.
In another embodiment of the invention:
the invention provides a crack growth detection method based on historical crack data, which comprises the following steps:
a historical map acquisition stage:
s1, acquiring a road surface image in real time through an image sensing module arranged on a map acquisition vehicle, and synchronously acquiring positioning information and inertial navigation information through a positioning module and an inertial navigation module;
s2, preprocessing the collected road surface image to obtain a corresponding gray level image; preprocessing the positioning information and inertial navigation information to obtain position information corresponding to the gray level image;
s3, enabling each gray image to correspond to the position information one by one, and enabling each gray image to be at a fixed interval by combining the actual vehicle speed and the acquisition frequency of the image sensing module to obtain a map set;
a positioning stage:
s4, acquiring a current road image in real time through an image sensing module arranged on a vehicle to be positioned, synchronously acquiring current positioning information through a positioning module, and extracting a plurality of similar images close to the current road image in a map set by comparing the current positioning information with position information in the map set;
s5, preprocessing the current road surface image to obtain a current gray level image; performing image feature extraction on the current gray level image and the similar image to obtain feature points which accord with feature uniqueness, time invariance, feature translation and rotation invariance and serve as pavement fingerprint information, and obtaining a closest pavement image according to the pavement fingerprint information;
and S6, calculating to obtain an accurate positioning result of the current road surface image according to the relative position relationship between the nearest road surface image and the fingerprint information in the current road surface image.
Further, the method for extracting image features in step S5 of the present invention is:
s51, detecting image characteristic points: detecting angular points through Harris algorithm, namely comparing the difference value of pixel points on a circle formed by a pixel point P in an image and a plurality of surrounding points, wherein the formula is as follows:
Figure BDA0002080340020000201
Figure BDA0002080340020000202
wherein, I (x) is the gray value of the current pixel point, I (P) is the gray value of the pixel point P, epsilon is a set threshold, the sum of the response function values of a plurality of points is N, and when N is greater than the judgment standard, the point is an image feature point;
determining the main direction of the feature points according to the direction from the feature points to the centroid:
Figure BDA0002080340020000203
Figure BDA0002080340020000204
θ＝arctan(m 10 ,m 01 )
wherein m is pq The moment of the image block is represented by (p, q epsilon (0, 1)), I (x, y) represents a gray value at the coordinate (x, y), and theta is an included angle between the centroid and the characteristic point;
s52, describing feature points:
selecting n pairs of features on the feature points to form a matrix s:
Figure BDA0002080340020000211
the size of the matrix s is 2 x 2n, where x i ,y i (i belongs to (1, n)) represents the coordinates of the ith feature pair in X and Y axes, an affine transformation matrix R is obtained by utilizing the directions from feature points to the centroid, and a new description matrix s is obtained by utilizing the matrix R for calculation θ The formula is as follows:
Figure BDA0002080340020000212
combining with BRIEF descriptor to obtain ORB feature descriptor:
g n (P,θ)＝f n (P)|(x i ,y i )∈M θ
wherein n takes the value of 256;
s53, feature point matching: calculating the Hamming distances of all feature descriptors of the current gray image to be positioned and the similar image according to the obtained 256-dimensional ORB feature descriptors, comparing the Hamming distances to realize coarse matching of feature points, and removing wrong matching by using a RANSAC algorithm so as to find the same feature points of the current gray image to be positioned and the similar image, wherein the matched feature points are used as pavement fingerprint information; and by comparing the number of the same characteristic points, finding out a closest road surface image from the roughly matched similar images, wherein the number of the characteristic points matched with the image to be positioned is the largest, thereby realizing the precise matching of the image to be positioned by utilizing the fingerprint information.
Further, the method for calculating the accurate positioning result of the current road surface image in step S6 of the present invention comprises:
the relative position relation of the two images is obtained by utilizing the fingerprint information of the current road surface image to be positioned and the similar image, and the position information of the current road surface image to be positioned is calculated by utilizing the position and the relative position relation of the similar image, so that the vehicle degree-magnitude positioning is realized.
Further, in step S2 of the present invention, the method for preprocessing the positioning information and the inertial navigation information to obtain the position information corresponding to the gray-scale image includes: and converting the GPS data and the inertial navigation data into position information represented by longitude and latitude.
Further, the fixed interval of each image in the map set in step S3 of the present invention is 0.5m.
It will be understood that modifications and variations can be made by persons skilled in the art in light of the above teachings and all such modifications and variations are intended to be included within the scope of the invention as defined in the appended claims.

Claims (9)

1. A pavement crack growth detection method based on historical crack data is characterized by comprising the following steps:
1) Acquiring a current pavement crack image in real time, and synchronously acquiring positioning information corresponding to the current pavement crack image;
2) Initial positioning: according to the current positioning information and the position information in the historical map data, extracting historical image data within a threshold distance from the current positioning information from the historical road surface image data, and selecting a plurality of similar images of the current road surface image from the historical road surface image data;
3) Image level localization: finding a closest road surface image from the roughly matched similar images in the step 2), wherein the method specifically comprises the following steps:
3.1 Gray-scale processing the image;
3.2 Calculating the Hamming distances of all feature descriptors of the current gray image and the similar image, comparing the Hamming distances to realize coarse matching of feature points, and then finding out the same feature points of the current gray image and the similar image, wherein the matched feature points are used as pavement fingerprint information;
3.3 By comparing the number of the same feature points, a closest road surface image is found in the rough matching similar images;
4) Pixel level positioning: mapping the historical cracks which are closest to the marks in the pavement image into the current pavement crack image;
5) And (3) carrying out RGM-based mapping crack analysis on the current pavement crack image, and detecting all pixels belonging to newly grown cracks in the collected crack image.
2. The method for detecting the crack growth of the road surface based on the historical crack data as claimed in claim 1, wherein the method for acquiring the feature descriptors in the step 3.2) is as follows:
3.2.1 Detect image feature points: detecting angular points as image characteristic points through a Harris algorithm;
3.2.2 Add direction information: adding direction information to the extracted characteristic points to enable the directions of the extracted characteristic points to be unchanged, wherein the directions are obtained by calculating entropy values of pixel blocks through moments of images in a circular window;
3.2.3 ORB feature point matching:
selecting n pairs of features on the feature points to form a mapping matrix s, wherein the size of the matrix s is 2X 2n, the elements of the matrix s are the coordinates of each feature pair on the X axis and the Y axis, then obtaining an affine transformation matrix R by utilizing the directions from the feature points to the centroid, and obtaining a new description matrix s by utilizing the matrix R for calculation θ And combining the BRIEF descriptor to obtain the ORB feature descriptor.
3. The road surface crack growth detection method based on historical crack data of claim 2, wherein the method for judging the image characteristic points in the step 3.2.1) is as follows: comparing the difference values of pixel points on a circular window formed by one pixel point P in the image and a plurality of surrounding points, wherein the sum of the difference values of the plurality of points is N, and when N is greater than a judgment standard, judging the point as an image feature point;
Figure FDA0003931853710000031
Figure FDA0003931853710000032
wherein, I (x) is the gray value of the current pixel point, I (P) is the gray value of the pixel point P, epsilon is the set threshold, circle (P) is the radiation range on the circular window around the pixel point P.
4. The method for detecting road surface crack growth based on historical crack data of claim 1, wherein the pixel-level positioning in the step 4) is mapping marked historical crack pixels into the current crack image.
5. The method for detecting the crack growth on the road surface based on the historical crack data as claimed in claim 2, wherein the pixel-level positioning in the step 4) is to find an H matrix of two images through image feature points matched by an ORB, and then map the marked historical crack pixels into the current crack image by using the H matrix.
6. The method for detecting the crack growth on the road surface based on the historical crack data according to claim 5, wherein the step 4) of the crack mapping based on the multi-scale positioning comprises the following specific steps:
assuming that the road is a plane, under the pinhole camera model, the homography matrix can be used to describe the basic geometric shape, and then two linear constraints on the homography matrix are known, and the historical crack label is mapped to the query image through the following relationship:
Figure FDA0003931853710000033
wherein n is the number of historical crack label pixels, [ u' i y′ i ] T Coordinates for mapping fracture data in historical data, [ u ] u i y i ] T The coordinates of the historical crack label pixels.
7. The method for detecting the crack growth of the road surface based on the historical crack data as claimed in claim 1, wherein the step 5) of analyzing the historical crack based on the RGM comprises the following specific steps:
5.1 Map crack gray value distribution analysis: taking the query crack labels mapped in the above steps as 'ideal' initial seed points, and starting from the seed points, expanding the region by searching for adjacent points with similar attributes to the seed points, wherein the attributes comprise color and intensity; calculating pixel values from the mapping corresponding relation on the query image through the mapping labels, and drawing an image intensity distribution histogram of the mapping crack data by utilizing a statistical principle;
5.2 The intensity distribution of the mapped crack pixels is represented by a gaussian model:
a Gaussian model is established to represent the distribution, the distribution characteristics are used as operators, and the corresponding calculation formula of the mean value and the standard deviation is as follows:
Figure FDA0003931853710000041
Figure FDA0003931853710000042
wherein N is the number of the pixels of the mapping crack label;
5.3 Growth crack analysis: when the intensity of a certain point in the image satisfies the following condition, the point is divided into cracks:
I(p μ ,p v )∈[0,ω+λσ]
in the formula, λ is a constant value determined according to the actual application.
8. The road surface crack growth detection method based on historical crack data of claim 1, wherein the positioning information in the step 1) is GPS positioning information.
9. The road surface crack growth detection method based on historical crack data of claim 3, wherein the number of the pixel points on the circular window around the pixel point P is 12 or 16.
CN201910469277.1A 2019-05-31 2019-05-31 Pavement crack growth detection method based on historical crack data Active CN110197157B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910469277.1A CN110197157B (en) 2019-05-31 2019-05-31 Pavement crack growth detection method based on historical crack data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910469277.1A CN110197157B (en) 2019-05-31 2019-05-31 Pavement crack growth detection method based on historical crack data

Publications (2)

Publication Number Publication Date
CN110197157A CN110197157A (en) 2019-09-03
CN110197157B true CN110197157B (en) 2023-03-24

Family

ID=67753569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910469277.1A Active CN110197157B (en) 2019-05-31 2019-05-31 Pavement crack growth detection method based on historical crack data

Country Status (1)

Country Link
CN (1) CN110197157B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112669453B (en) * 2020-11-30 2023-05-02 南方咨询(湖北)有限公司 Municipal road detecting system
CN113129282A (en) * 2021-04-16 2021-07-16 广东韶钢松山股份有限公司 Belt abnormality determination method, device, equipment and storage medium
CN113506292B (en) * 2021-07-30 2022-09-20 同济大学 Structure surface crack detection and extraction method based on displacement field
CN114359147A (en) * 2021-12-03 2022-04-15 深圳大学 Crack detection method, crack detection device, server and storage medium
CN114898107B (en) * 2022-07-01 2022-12-02 深之蓝海洋科技股份有限公司 Crack re-identification method and device
CN117037105B (en) * 2023-09-28 2024-01-12 四川蜀道新能源科技发展有限公司 Pavement crack filling detection method, system, terminal and medium based on deep learning
CN117890900A (en) * 2024-01-18 2024-04-16 中建三局信息科技有限公司 Radar positioning method, radar positioning device, electronic equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016172827A1 (en) * 2015-04-27 2016-11-03 武汉武大卓越科技有限责任公司 Stepwise-refinement pavement crack detection method
CN106960591A (en) * 2017-03-31 2017-07-18 武汉理工大学 A kind of vehicle high-precision positioner and method based on road surface fingerprint
CN108229562A (en) * 2018-01-03 2018-06-29 重庆亲禾智千科技有限公司 It is a kind of to obtain the method for the specific failure modes situation in road surface

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016172827A1 (en) * 2015-04-27 2016-11-03 武汉武大卓越科技有限责任公司 Stepwise-refinement pavement crack detection method
CN106960591A (en) * 2017-03-31 2017-07-18 武汉理工大学 A kind of vehicle high-precision positioner and method based on road surface fingerprint
CN108229562A (en) * 2018-01-03 2018-06-29 重庆亲禾智千科技有限公司 It is a kind of to obtain the method for the specific failure modes situation in road surface

Also Published As

Publication number Publication date
CN110197157A (en) 2019-09-03

Similar Documents

Publication Publication Date Title
CN110197157B (en) Pavement crack growth detection method based on historical crack data
CN111028277B (en) SAR and optical remote sensing image registration method based on pseudo-twin convolution neural network
CN106778823B (en) Automatic identification method for reading of pointer instrument
CN110647850A (en) Automatic lane deviation measuring method based on inverse perspective principle
CN108180912B (en) Mobile robot positioning system and method based on hybrid navigation band
Yang et al. Fast and accurate vanishing point detection and its application in inverse perspective mapping of structured road
CN109461132B (en) SAR image automatic registration method based on feature point geometric topological relation
CN107145890B (en) A kind of pointer dashboard automatic reading method under remote multi-angle of view environment
CN106407924A (en) Binocular road identifying and detecting method based on pavement characteristics
CN103606170B (en) Streetscape image feature based on colored Scale invariant detects and matching process
CN110084205A (en) One kind being based on improved object-oriented Classification of High Resolution Satellite Images method
CN112396612B (en) Vector information assisted remote sensing image road information automatic extraction method
CN112070756B (en) Three-dimensional road surface disease measuring method based on unmanned aerial vehicle oblique photography
CN107862319B (en) Heterogeneous high-light optical image matching error eliminating method based on neighborhood voting
US11941878B2 (en) Automated computer system and method of road network extraction from remote sensing images using vehicle motion detection to seed spectral classification
CN112800938B (en) Method and device for detecting occurrence of side rockfall of unmanned vehicle
CN111288890A (en) Road sign dimension and height automatic measurement method based on binocular photogrammetry technology
CN103913166A (en) Star extraction method based on energy distribution
CN110070012A (en) A kind of refinement extracted applied to remote sensing image road network and global connection method
Liu et al. Image-translation-based road marking extraction from mobile laser point clouds
CN117274627A (en) Multi-temporal snow remote sensing image matching method and system based on image conversion
CN115222819A (en) Camera self-calibration and target tracking method based on multi-mode information reference in airport large-range scene
CN108875454A (en) Traffic sign recognition method, device and vehicle
Sui et al. Automatic image registration based on shape features and multi-scale image segmentation
Marques et al. Crater delineation by dynamic programming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant