CN110084844B - Airport pavement crack detection method based on depth camera - Google Patents
Airport pavement crack detection method based on depth camera Download PDFInfo
- Publication number
- CN110084844B CN110084844B CN201910338449.1A CN201910338449A CN110084844B CN 110084844 B CN110084844 B CN 110084844B CN 201910338449 A CN201910338449 A CN 201910338449A CN 110084844 B CN110084844 B CN 110084844B
- Authority
- CN
- China
- Prior art keywords
- curved surface
- road surface
- grid
- depth
- crack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000012216 screening Methods 0.000 claims abstract description 8
- 239000013598 vector Substances 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000001419 dependent effect Effects 0.000 claims 1
- 238000005286 illumination Methods 0.000 abstract description 4
- 230000007613 environmental effect Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000003708 edge detection Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004392 development of vision Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000009659 non-destructive testing Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30132—Masonry; Concrete
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
Abstract
Provided is an airport pavement crack detection method based on a depth camera. The method comprises the steps of acquiring a depth image of an airport pavement by using a depth camera, and dividing the image into a plurality of grids; expanding the grids; constructing a road surface curved surface model for the expanded grid; connecting the road surface curved surface models of the grids to obtain an integral curved surface model of the road surface in the depth image; calculating a difference value between the depth image and the integral surface model to obtain candidate crack pixel points; and screening candidate crack pixel points to obtain a real crack area. The method has the advantages that: the method is applied to airport pavement crack detection, manual operation is replaced by a high-performance automatic airport pavement detection method, detection precision and working efficiency can be improved, and safety performance of the airport pavement is improved. The method is not influenced by illumination change, and has stronger robustness to environmental noise. The modeling of the pavement structure is more accurate, so that the crack detection accuracy based on pavement model reconstruction is higher.
Description
Technical Field
The invention belongs to the technical field of nondestructive testing, and particularly relates to an airport pavement crack detection method based on a depth camera.
Background
The airport runway is the most important infrastructure for airplane flight, and due to the increase of flight frequency and the damage of natural environment, a plurality of airport runway surfaces are damaged in different degrees, so that great potential safety hazards are caused, and cracks are the main defects of the airport runway surfaces. Therefore, the crack detection technology of the airport runway is increasingly emphasized.
At present, the crack detection work of the airport runways mainly depends on a manual inspection mode of manual observation and manual recording. However, manual inspection has many problems of poor precision, easy omission, strong subjectivity, low efficiency and the like, and a high-performance automatic airport runway defect detection method is urgently to be developed to replace manual operation, so that the working efficiency and the safety performance are improved. With the rapid development of vision sensor technology and pattern recognition and other related technologies, some researchers have started to pay attention to crack detection technology based on computer vision. For the detection of road and bridge pavement cracks with certain similarity to the road and bridge pavement cracks, related researches are mostly based on visible light sensors at present and have achieved some achievements, and the problem of robustness in a complex environment is the biggest difficulty faced by the existing researches. The road surface crack detection method based on the visible light camera can be divided into four types: grayscale threshold based methods, edge detection based methods, machine learning methods, and morphology methods. The method based on the gray threshold is sensitive to noise, and particularly, the crack detection effect is very unreliable when the illumination condition is poor. The main disadvantage of the method based on edge detection is that edge detection can only extract discontinuous crack features because the connectivity of cracks is not considered, and the method often fails in a scene with low contrast and strong noise interference. The machine learning method cannot guarantee that the global crack information of the whole image is extracted, and in addition, a large number of accurately marked samples are needed in the learning process, and the requirement is difficult to realize in the application of obvious illumination and scene change. The crack detection effect based on morphology is greatly influenced by parameter selection, and has difficulty in practical application. Most of the research already, such as the above mentioned work, is carried out on roads or bridges. Although the document also states that the method for detecting the cracks of the pavement surface of the highway can be also used for the airfield runway, no practical experimental verification is seen. Compared with roads and bridge pavements, the airport runway generates obvious oil stains, rubber residual traces and the like due to frequent take-off and landing of airplanes, the contrast between cracks and the background is very low due to runway materials, in addition, the runway pavement detection can be only carried out under the condition of an artificial light source at night, the crack characteristics are usually very small, and the factors make the visual detection of the airport runway pavement cracks very difficult. In recent years, a depth sensor is used for detecting the defects of the pavement, but the depth sensor mainly aims at detecting pits on a road, and the pavement model is often approximately regarded as a plane, so that the detection effect is not ideal.
Disclosure of Invention
In order to solve the above problems, an object of the present invention is to provide a depth camera-based airport pavement crack detection method.
In order to achieve the above purpose, the airport pavement crack detection method based on the depth camera provided by the invention comprises the following steps in sequence:
step 1) acquiring a depth image of an airport pavement by using a depth camera, and dividing the acquired depth image into a plurality of grids;
step 2) expanding each grid;
step 3) constructing a road surface curved surface model for the expanded grid;
step 4) connecting the road surface curved surface models of the grids to obtain an integral curved surface model of the road surface in the depth image;
step 5) calculating the difference value between the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) to obtain candidate crack pixel points;
and 6) screening the candidate crack pixel points to obtain a real crack area.
In step 1), the method for acquiring the depth image of the airport pavement by using the depth camera and then dividing the acquired depth image into a plurality of grids comprises the following steps:
firstly, a depth image of an airport pavement is collected by a depth camera, and then the collected depth image is divided into n grids Y with the size of k pixel points x k pixel points i Defining the grid area as Y i ,i＝1,2,…,n。
In step 2), the method for expanding each grid includes:
increasing the size of each grid to the range of k/2 pixel points around to make each expanded grid
The size of the grid area is 2k x 2k pixel points, and the expanded grid area is defined as->
i＝1,2,…,n。
In step 3), the method for constructing the road surface curved surface model for the expanded mesh comprises the following steps:
for expanded grid area
i =1,2, …, n, defining P (x, y, z) as being located in an expanded lattice @>
Wherein x is the abscissa of the pixel in the depth image, y is the ordinate of the pixel in the depth image, and z is the depth value of the road surface curved surface model at the coordinate (x, y);
establishing a curved surface model of the runway surface for the curved surface of the airport runway surface by a cubic surface equation,
let A = [ a ] 0 ,a 1 ,a 2 ,a 3 ,a 4 ,a 5 ,a 6 ,a 7 ,a 8 ,a 9 ] T ，H＝[1,x,y,x 2 ,xy,y 2 ,x 3 ,x 2 y,xy 2 ,y 3 ]And then:
z＝H·A
wherein A is the parameter vector of the road surface curved surface model to be solved, and H is the expanded grid Y i * Independent variable vectors consisting of horizontal and vertical coordinates of the middle pixel points; the invention adopts RANSAC algorithm framework and least square method to estimate the road surface curved surface model, and the specific method is as follows: in the RANSAC algorithm framework, 9 pixel points are randomly selected each time to estimate the road surfaceSurface model, let Z c ＝[z 1 ,z 2 ,z 3 ,...,z c ] T Depth value vector representing c pixels, order
Representing an independent variable matrix consisting of coordinates of c pixel points; the system of linear equations for the parameter vector of the unknown road surface curved surface model is:
Z c ＝X c ·A
then, solving the linear equation set by adopting a least square method to obtain the parameter vector of the road surface curved surface model:
then using the parameter vector of the road surface curved surface model
Evaluating the expanded grid->
Depth values for all coordinates within:
wherein
Representing data combined by expanded grids>
An independent variable matrix formed by coordinates of all the internal pixel points; let Z 0 Representing the original depth value, the shortest distance between the road surface curved surface model and the original depth value can be calculated:
if the distance from a certain pixel point to the curved surface of the road surface is less than a distance threshold value T i If yes, the pixel point is judged as an inner point, otherwise, the pixel point is judged as an outer point; finally, selecting the curved surface model of the road surface with the largest number of obtained interior points as a real model, and using the interior point set formed by the interior points to re-optimize and estimate the expanded grid by using a least square method
The curve surface model parameter vector of the road surface>
In step 4), the method for connecting the road surface curved surface models of the meshes to obtain the overall curved surface model of the road surface in the depth image includes:
taking out the original grid in the road surface curved surface fitting model of the expanded grid obtained in the step 3) as a road surface curved surface model of the original grid; splicing respective road surface curved surface models together according to the positions of the original grids to form an integral curved surface model of the road surface in the depth image, wherein the concrete method comprises the following steps: repeating the step 3) until the expanded grid area
i =1,2, …, n are all calculated, and a track surface curved surface model based on each grid is obtained>
i =1,2.., n, based on the road surface curve model for each grid->
And forming an independent variable vector by the pixel point coordinates of the depth image:
wherein i tableShowing the ith grid, showing the jth pixel point in the ith grid by j, and calculating each coordinate in the depth image
Depth value of the road surface curved surface:
in step 5), the method for calculating the difference between the depth image obtained in step 1) and the overall curved surface model obtained in step 4) to obtain the candidate crack pixel point includes:
making a difference between the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) according to the corresponding positions of pixel points, and taking an absolute value, wherein d (x, y) is an absolute value of the difference at the pixel points (x, y); if the absolute value d (x, y) is greater than a set threshold T d If not, the pixel point (x, y) is regarded as a candidate crack pixel point, and if not, the pixel point is a non-crack pixel.
In step 6), the method for obtaining the real crack region by screening the candidate crack pixel points comprises the following steps:
dividing the obtained candidate crack pixel points into a plurality of connected regions according to connectivity, and screening out a real crack region by calculating the area, the length and the length-width ratio of each connected region, wherein the specific method comprises the following steps: firstly, extracting a framework of each connected region, defining the number of framework pixel points as the length of the connected region, marking the length as l, then calculating the distance from each pixel point on the framework to the edge of the connected region along the normal direction of the pixel point, and taking the average value of all the distances as the width of the connected region, and marking the width as w; recording the total number of pixel points of the connected region as m; if the following conditions are simultaneously satisfied: m is a unit of>T m And l>T l And is
Marking the connected region as a real crack region, otherwise marking the connected region as a non-crack region and deleting, wherein T m 、T l And T r Is a set threshold.
In step 5), the threshold value T d Indicating the minimum depth of crack detectable, which is set at 5 mm according to the detection requirements of the airport pavement.
In step 7), the threshold value T m The value is an empirical value and ranges from 30 to 60; threshold value T l The value is an empirical value and ranges from 20 to 30; threshold value T r The range of the empirical value is not less than 7.
Compared with the prior art, the airport pavement crack detection method based on the depth camera has the following advantages: (1) the method is applied to the detection of the airport pavement cracks, and the high-performance automatic airport pavement detection method replaces manual operation, so that the detection precision and the working efficiency can be improved, and the safety performance of the airport pavement is further improved. (2) The invention is not influenced by illumination change and has stronger robustness to environmental noise. (3) The method of the invention has more accurate modeling of the pavement structure, thereby leading the crack detection accuracy based on pavement model reconstruction to be higher.
Drawings
FIG. 1 is a flow chart of a depth camera based airport pavement crack detection method provided by the present invention;
FIG. 2 is a schematic diagram of the main steps of the present invention;
fig. 3 shows the airport pavement crack detection results.
Detailed Description
The airport pavement crack detection method based on the depth camera provided by the invention is described in detail below with reference to the accompanying drawings and specific embodiments.
The depth camera operates by emitting successive near infrared pulses to the target scene and then receiving the light pulses reflected back by the object with a sensor. By comparing the phase difference between the emitted light pulse and the light pulse reflected by the object, the transmission delay between the light pulses can be calculated, the distance between the object and the emitter can be further obtained, and finally a depth image can be obtained. The gray value of each pixel point in the depth image can be used for representing the distance between a certain point in the target scene and the depth camera. The crack regions on the airport pavement appear in the depth image to have a gray value greater than the gray value of the airport pavement. However, in order to increase the friction, the airport runway may increase the roughness of the surface of the runway in the design, so that the gray value of a part of the airport runway surface area in the acquired depth image is similar to or even lower than the deep gray value of the crack area, and the crack is difficult to be directly extracted from the depth image. The method solves the problem of airport pavement crack detection by using a depth camera.
As shown in fig. 1, the depth camera-based airport pavement crack detection method provided by the invention comprises the following steps performed in sequence:
step 1) acquiring a depth image of an airport pavement by using a depth camera, and dividing the acquired depth image into a plurality of grids;
firstly, a depth image of an airport pavement is collected by a depth camera, and then the collected depth image is divided into n grids Y with the size of k pixel points x k pixel points i As shown in fig. 2 (a). If the value of k is too large, the consistency of the depth image gray value model in the grid is poor, and subsequent surface fitting and crack detection are not facilitated; if the value of k is too small, crack pixel points may occupy most of the grid area, so that subsequent road surface curved surface fitting cannot be performed. According to multiple experimental tests, k =100 is taken in the experiment of the invention. Defining the grid area as Y i ,i＝1,2,…,n。
Step 2) expanding each grid;
because the grids are independent from each other, there is no connection between the road surface curved surface models established by each grid, which results in discontinuous road surface curved surface models of adjacent grids at the edge position and reduced precision, therefore, each grid needs to be expanded to strengthen the connection between the adjacent grids and improve the precision of the road surface curved surface models at the edge area of the grids. The specific method for expanding each grid is as follows: increasing the size of each grid to the range of k/2 pixel points around to make each expanded grid
The size of (2 k) is 2k pixels, as shown in fig. 2 (b). The expanded grid area is defined as->
i＝1,2,…,n。
Step 3) constructing a road surface curved surface model for the expanded grid;
for expanded grid area
i =1,2, …, n, defining P (x, y, z) as being located in the expanded grid ≧ H>
Wherein x is an abscissa of a pixel in the depth image, y is an ordinate of a pixel in the depth image, and z is a depth value of the curved surface model of the road surface at the coordinates (x, y).
As shown in fig. 2 (c), a curved surface model of the runway surface is established for the curved surface of the airport runway surface by a cubic surface equation,
let A = [ a ] 0 ,a 1 ,a 2 ,a 3 ,a 4 ,a 5 ,a 6 ,a 7 ,a 8 ,a 9 ] T ，H＝[1,x,y,x 2 ,xy,y 2 ,x 3 ,x 2 y,xy 2 ,y 3 ]And then:
z＝H·A
wherein A is the parameter vector of the road surface curved surface model to be solved, and H is the expanded grid
And the independent variable vector is formed by the horizontal and vertical coordinates of the middle pixel point. Due to the expanded grid->
Not only includes the pavement coordinates, but also possibly includes the coordinates of a crack area, therefore, the invention adopts RANSAC algorithm framework and least square method to estimate the pavement curved surfaceThe model comprises the following specific steps: in the RANSAC algorithm framework, randomly selecting 9 pixel points each time to estimate the curved surface model of the road surface, and enabling Z to be c ＝[z 1 ,z 2 ,z 3 ,...,z c ] T A depth value vector representing c pixel points, make->
Representing an argument matrix consisting of the coordinates of the c pixels. The system of linear equations for the parameter vector of the unknown road surface curved surface model is:
Z c ＝X c ·A
then, solving the linear equation set by adopting a least square method to obtain the parameter vector of the road surface curved surface model:
then, the parameter vector of the road surface curved surface model is utilized
Evaluating expanded grids>
Depth values for all coordinates within:
wherein
Indicating that is being conditioned by the expanded grid>
And the coordinates of all the pixels in the independent variable matrix form the independent variable matrix. Let Z 0 Representing the original depth value, the shortest distance between the road surface curved surface model and the original depth value can be calculated:
if the distance from a certain pixel point to the curved surface of the road surface is less than a distance threshold value T i If yes, the pixel point is judged as an inner point, otherwise, the pixel point is judged as an outer point; finally, selecting the curved surface model of the road surface with the largest number of obtained interior points as a real model, and using the interior point set formed by the interior points to re-optimize and estimate the expanded grid by using a least square method
The curve surface model parameter vector of the road surface>
Distance threshold T i The actual depth of the crack and the depth difference of the rough texture designed on the road surface need to be comprehensively considered, and the value is 5 mm.
Step 4) connecting the road surface curved surface models of the grids to obtain an integral curved surface model of the road surface in the depth image;
taking out the original grid in the road surface curved surface fitting model of the expanded grid obtained in the step 3) as a road surface curved surface model of the original grid; according to the position of the original mesh, the respective road surface curved surface models are spliced together to form an overall curved surface model of the road surface in the depth image, as shown in fig. 2 (d). The specific method comprises the following steps: repeating the step 3) until the expanded grid area
i =1,2, …, n are all calculated, and a track surface curved surface model based on each grid is obtained>
i =1,2.., n, based on the road surface curve model for each grid->
And the coordinates of the pixel points of the depth image form an independent variable vector>
Wherein i represents the ith grid, j represents the jth pixel point in the ith grid, and each coordinate in the depth image is calculated
Depth value of the road surface curved surface:
step 5) calculating the difference value between the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) to obtain candidate crack pixel points;
because the distance between the road surface area point and the road surface curved surface model and the distance between the crack area point and the road surface curved surface model in the actual data have obvious difference, the depth image obtained in the step 1) and the overall curved surface model obtained in the step 4) are subjected to subtraction according to the corresponding positions of the pixel points and an absolute value is obtained, and d (x, y) is an absolute value of the difference at the pixel point (x, y). If the absolute value d (x, y) is greater than a set threshold T d If not, the pixel point (x, y) is regarded as a candidate crack pixel point, and if not, the pixel point is a non-crack pixel. Threshold value T d Indicating the minimum depth of crack detectable, which is set at 5 mm according to the detection requirements of the airport pavement. As shown in fig. 2 (e).
Step 6) screening the candidate crack pixel points to obtain a real crack area;
dividing the obtained candidate crack pixel points into a plurality of connected regions according to connectivity, and screening out a real crack region by calculating the area, length and length-width ratio of each connected region, as shown in fig. 2 (f). The specific method comprises the following steps: firstly, extracting the skeleton of each connected region, defining the number of skeleton pixel points as the length of the connected region, recording as l, and then calculating the distance from each pixel point on the skeleton to the connected region along the normal direction thereofTaking the average value of all the distances as the width of the connected region and recording as w; and recording the total number of the pixel points of the connected region as m. If the following conditions are simultaneously satisfied: m is>T m And l>T l And is
Marking the connected region as a real crack region, otherwise marking the connected region as a non-crack region and deleting, wherein T m 、T l And T r Is a set threshold. Threshold value T m The value range is 30-60 for empirical value; threshold value T l The value range is 20-30 for empirical value; threshold value T r The value range is an empirical value and is not less than 7.
The effect of the airport pavement crack detection method based on the depth camera can be further illustrated by the following experimental results. The inventor acquires 42 depth images of the airport pavement to verify the method, and the experimental result is shown in fig. 3, wherein the left image of the same row is the depth image of the airport pavement, and the right image is the crack region extraction result.
Claims (8)
1. A depth camera-based airport pavement crack detection method is characterized by comprising the following steps: the airport pavement crack detection method based on the depth camera comprises the following steps in sequence:
step 1) acquiring a depth image of an airport pavement by using a depth camera, and dividing the acquired depth image into a plurality of grids;
step 2) expanding each grid of the grids;
step 3) constructing a road surface curved surface model for the expanded grid;
step 4) connecting the extended road surface curved surface model of the grid to obtain an integral curved surface model of the road surface in the depth image;
step 5) calculating the difference value between the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) to obtain candidate crack pixel points;
step 6) screening the candidate crack pixel points to obtain a real crack area;
in step 3), the method for constructing the road surface curved surface model for the expanded grid includes:
for expanded grid area Y i * And =1,2, …, n, define P (x, Y, z) as located in expanded grid Y i * Wherein x is the abscissa of the pixel in the depth image, y is the ordinate of the pixel in the depth image, and z is the depth value of the road surface curved surface model at the coordinate (x, y);
establishing a curved surface model of the runway surface for the curved surface of the airport runway surface by a cubic surface equation,
let A = [ a ] 0 ,a 1 ,a 2 ,a 3 ,a 4 ,a 5 ,a 6 ,a 7 ,a 8 ,a 9 ] T ，H＝[1,x,y,x 2 ,xy,y 2 ,x 3 ,x 2 y,xy 2 ,y 3 ]And then:
z＝H·A
wherein A is the parameter vector of the road surface curved surface model to be solved, and H is the expanded grid Y i * Independent variable vectors consisting of horizontal and vertical coordinates of the middle pixel points; the invention adopts RANSAC algorithm framework and least square method to estimate the road surface curved surface model, and the specific method is as follows: in the RANSAC algorithm framework, randomly selecting 9 pixel points each time to estimate the curved surface model of the road surface, and enabling Z to be c ＝[z 1 ,z 2 ,z 3 ,...,z c ] T Depth value vector representing c pixels, order
Representing an independent variable matrix consisting of coordinates of c pixel points; the system of linear equations for the parameter vector of the unknown road surface curved surface model is:
Z c ＝X c ·A
then solving the linear equation set by adopting a least square method to obtain the parameter vector of the road surface curved surface model:
then using the parameter vector of the road surface curved surface model
Obtaining expanded grid Y i * Depth values for all coordinates within:
wherein
Representation by expanded grid Y i * An independent variable matrix formed by coordinates of all pixel points in the matrix; let Z 0 Representing the original depth value, the shortest distance between the road surface curved surface model and the original depth value can be calculated:
if the distance from a certain pixel point to the curved surface of the road surface is less than the distance threshold value T i Judging the pixel point as an inner point, otherwise, judging the pixel point as an outer point; and finally, selecting the road surface curved surface model with the largest number of obtained interior points as a real model, and re-optimizing and estimating the expanded grid Y by using a least square method by utilizing an interior point set formed by all interior points on the road surface curved surface model with the largest number of interior points i * Road surface curved surface model parameter vector
2. The depth camera-based airport pavement crack detection method of claim 1, wherein: in step 1), the method for acquiring the depth image of the airport pavement by using the depth camera and then dividing the acquired depth image into a plurality of grids comprises the following steps:
firstly, a depth image of an airport pavement is collected by a depth camera, and then the collected depth image is divided into n grids Y with the size of k pixel points x k pixel points i Defining the grid area as Y i ,i＝1,2,…,n。
3. The depth camera-based airport pavement crack detection method of claim 1, wherein: in step 2), the method for expanding each grid of the multiple grids comprises:
respectively increasing the size of each grid to the surrounding by k/2 pixel point ranges to ensure that each expanded grid Y i * The size of the grid is 2k x 2k pixel points, and the expanded grid area is defined as Y i * ,i＝1,2,…,n。
4. The depth camera-based airport pavement crack detection method of claim 1, wherein: in step 4), the method for connecting the road surface curved surface model of the expanded mesh to obtain the overall curved surface model of the road surface in the depth image is as follows:
taking out the original grid in the road surface curved surface fitting model of the expanded grid obtained in the step 3) as a road surface curved surface model of the original grid; splicing respective road surface curved surface models together according to the positions of the original grids to form an integral curved surface model of the road surface in the depth image, wherein the concrete method comprises the following steps: repeating the step 3) until the expanded grid area Y i * I =1,2, …, n are all calculated, and the road surface curved surface model of each grid is obtained
According to the curved surface model of the road surface of each grid>
And forming an independent variable vector by the pixel point coordinates of the depth image:
wherein i represents the ith grid, j represents the jth pixel point in the ith grid, and each coordinate in the depth image is calculated
Depth value of the road surface curved surface:
5. the depth camera-based airport pavement crack detection method of claim 1, wherein: in step 5), the method for calculating the difference between the depth image obtained in step 1) and the overall curved surface model obtained in step 4) to obtain the candidate crack pixel point includes:
making a difference between the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) according to the corresponding positions of the pixel points, and taking an absolute value, wherein d (x, y) is an absolute value of a difference value of the depth image obtained in the step 1) and the integral curved surface model obtained in the step 4) at the pixel points (x, y); if the absolute value d (x, y) is greater than a set threshold T d If not, the pixel point (x, y) is regarded as a candidate crack pixel point, and if not, the pixel point is a non-crack pixel.
6. The depth camera-based airport pavement crack detection method of claim 1, wherein: in step 6), the method for screening the candidate crack pixel points to obtain the real crack region includes:
dividing the obtained candidate crack pixel points into a plurality of connected areas according to connectivity, and countingCalculating the area, length and length-width ratio of each connected region to screen out a real crack region, wherein the specific method comprises the following steps: firstly, extracting a skeleton of each connected region, defining the number of skeleton pixel points as the length of the connected region, marking as l, then calculating the distance from each pixel point on the skeleton to the edge of the connected region along the normal direction of the pixel point, and taking the average value of all the distances as the width of the connected region, and marking as w; recording the total number of pixel points of the connected region as m; if the following conditions are simultaneously satisfied: m > T m And l > T l And is
Marking the connected region as a real crack region, otherwise marking the connected region as a non-crack region and deleting, wherein T m 、T l And T r Is a set threshold.
7. The depth camera-based airport pavement crack detection method of claim 5, wherein: the threshold value T d Indicating the minimum depth of the crack to be detected, the threshold value T being dependent on the detection requirements of the airport pavement d Set to 5 mm.
8. The depth camera-based airport pavement crack detection method of claim 6, wherein: the threshold value T m The value is an empirical value and ranges from 30 to 60; threshold value T l The value is an empirical value and ranges from 20 to 30; threshold value T r The value range is an empirical value and is not less than 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910338449.1A CN110084844B (en) | 2019-04-25 | 2019-04-25 | Airport pavement crack detection method based on depth camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910338449.1A CN110084844B (en) | 2019-04-25 | 2019-04-25 | Airport pavement crack detection method based on depth camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110084844A CN110084844A (en) | 2019-08-02 |
| CN110084844B true CN110084844B (en) | 2023-03-28 |
Family
ID=67416697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910338449.1A Active CN110084844B (en) | 2019-04-25 | 2019-04-25 | Airport pavement crack detection method based on depth camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110084844B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112053331B (en) * | 2020-08-28 | 2023-04-07 | 西安电子科技大学 | Bridge crack detection method based on image superposition and crack information fusion |
| CN112374081A (en) * | 2020-11-12 | 2021-02-19 | 泉州冰点科技有限公司 | Large-scale conveyer belt surface crack detection method and system |
| CN112666167B (en) * | 2020-12-22 | 2022-12-13 | 成都圭目机器人有限公司 | Method and device for evaluating FOD risk generated by cement concrete pavement crack |
| CN112726360B (en) * | 2020-12-24 | 2022-03-01 | 中铁建设集团基础设施建设有限公司 | Airport concrete pavement crack repairing method |
| CN116664582B (en) * | 2023-08-02 | 2023-10-27 | 四川公路桥梁建设集团有限公司 | Road surface detection method and device based on neural vision network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102183525A (en) * | 2011-01-20 | 2011-09-14 | 单宝华 | Bridge apparent state automatic detection device and method based on charge coupled device (CCD) array photographic technique |
| CN105113375A (en) * | 2015-05-15 | 2015-12-02 | 南京航空航天大学 | Pavement cracking detection system and method based on line structured light |
| CN105133471A (en) * | 2015-05-15 | 2015-12-09 | 南京航空航天大学 | Linear structured light pavement surface detection system-based pavement depth image production method |
| CA2914020A1 (en) * | 2014-12-10 | 2016-06-10 | Dassault Systemes | Texturing a 3d modeled object |
| CN107290346A (en) * | 2017-04-10 | 2017-10-24 | 南京航空航天大学 | Pavement crack detecting system based on two dimensional image and line structure optical sensor |
| CN107507170A (en) * | 2017-07-28 | 2017-12-22 | 中国民航大学 | A kind of airfield runway crack detection method based on multi-scale image information fusion |
| CN107633516A (en) * | 2017-09-21 | 2018-01-26 | 武汉武大卓越科技有限责任公司 | A kind of method and apparatus for identifying surface deformation class disease |
| CN108921076A (en) * | 2018-09-21 | 2018-11-30 | 南京信息工程大学 | Road face crack disease self-adaption constant false-alarm detection method based on image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013090830A1 (en) * | 2011-12-16 | 2013-06-20 | University Of Southern California | Autonomous pavement condition assessment |
-
2019
- 2019-04-25 CN CN201910338449.1A patent/CN110084844B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102183525A (en) * | 2011-01-20 | 2011-09-14 | 单宝华 | Bridge apparent state automatic detection device and method based on charge coupled device (CCD) array photographic technique |
| CA2914020A1 (en) * | 2014-12-10 | 2016-06-10 | Dassault Systemes | Texturing a 3d modeled object |
| CN105113375A (en) * | 2015-05-15 | 2015-12-02 | 南京航空航天大学 | Pavement cracking detection system and method based on line structured light |
| CN105133471A (en) * | 2015-05-15 | 2015-12-09 | 南京航空航天大学 | Linear structured light pavement surface detection system-based pavement depth image production method |
| CN107290346A (en) * | 2017-04-10 | 2017-10-24 | 南京航空航天大学 | Pavement crack detecting system based on two dimensional image and line structure optical sensor |
| CN107507170A (en) * | 2017-07-28 | 2017-12-22 | 中国民航大学 | A kind of airfield runway crack detection method based on multi-scale image information fusion |
| CN107633516A (en) * | 2017-09-21 | 2018-01-26 | 武汉武大卓越科技有限责任公司 | A kind of method and apparatus for identifying surface deformation class disease |
| CN108921076A (en) * | 2018-09-21 | 2018-11-30 | 南京信息工程大学 | Road face crack disease self-adaption constant false-alarm detection method based on image |
Non-Patent Citations (3)
| Title |
|---|
| 《基于结构光的路面裂缝检测关键技术研究》;朱春省;《工程科技II辑》;20180330;全文 * |
| 基于空间聚集特征的沥青路面裂缝检测方法;张德津等;《自动化学报》;20160315(第03期);全文 * |
| 基于结构光的路面裂缝检测方法研究;宋葵阳等;《机械制造与自动化》;20161220(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110084844A (en) | 2019-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110084844B (en) | Airport pavement crack detection method based on depth camera | |
| Kang et al. | Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning | |
| Wang et al. | Computational methods of acquisition and processing of 3D point cloud data for construction applications | |
| Wang et al. | Pavement crack image acquisition methods and crack extraction algorithms: A review | |
| Chen et al. | Automated extraction and evaluation of fracture trace maps from rock tunnel face images via deep learning | |
| WO2022147969A1 (en) | Airport pavement underground structure disease automatic detection method based on deep learning | |
| Yan et al. | Automated extraction of structural elements in steel girder bridges from laser point clouds | |
| CN110246130B (en) | Airport pavement crack detection method based on infrared and visible light image data fusion | |
| Ye et al. | Automatic pixel‐level crack detection with multi‐scale feature fusion for slab tracks | |
| Guldur | Laser-based structural sensing and surface damage detection | |
| Panella et al. | Deep learning and image processing for automated crack detection and defect measurement in underground structures | |
| CN111178392B (en) | Aero-engine hole detection image damage segmentation method based on deep neural network | |
| Vetrivel et al. | Segmentation of UAV-based images incorporating 3D point cloud information | |
| CN104318051A (en) | Rule-based remote-sensing automatic extraction system and method of wide-range water body information | |
| CN111985552A (en) | Method for detecting diseases of thin strip-shaped structure of airport pavement under complex background | |
| Che et al. | Fast edge detection and segmentation of terrestrial laser scans through normal variation analysis | |
| Zou et al. | Automatic segmentation, inpainting, and classification of defective patterns on ancient architecture using multiple deep learning algorithms | |
| CN115512247A (en) | Regional building damage grade assessment method based on image multi-parameter extraction | |
| Ikeno et al. | An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs | |
| Germanese et al. | Towards structural monitoring and 3d documentation of architectural heritage using UAV | |
| Zhang et al. | Automated fatigue crack detection in steel box girder of bridges based on ensemble deep neural network | |
| Cao | Research on crack detection of bridge deck based on computer vision | |
| Yu et al. | Road Surface Defect Detection—From Image-Based to Non-Image-Based: A Survey | |
| CN116958837A (en) | Municipal facilities fault detection system based on unmanned aerial vehicle | |
| Ya’acob et al. | Real-Time Pavement Crack Detection Based on Artificial Intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190802 Assignee: TIANJIN JIUYUE TECHNOLOGY Co.,Ltd. Assignor: CIVIL AVIATION University OF CHINA Contract record no.: X2024980001620 Denomination of invention: A crack detection method for airport pavement based on depth camera Granted publication date: 20230328 License type: Common License Record date: 20240130 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190802 Assignee: Zhongke Qingyan (Tianjin) Technology Co.,Ltd. Assignor: CIVIL AVIATION University OF CHINA Contract record no.: X2024980003284 Denomination of invention: A crack detection method for airport pavement based on depth camera Granted publication date: 20230328 License type: Common License Record date: 20240321 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190802 Assignee: FITOW (TIANJIN) DETECTION TECHNOLOGY CO.,LTD. Assignor: CIVIL AVIATION University OF CHINA Contract record no.: X2024980003591 Denomination of invention: A crack detection method for airport pavement based on depth camera Granted publication date: 20230328 License type: Common License Record date: 20240327 |