CN117218089A - Asphalt pavement structure depth detection method - Google Patents
Asphalt pavement structure depth detection method Download PDFInfo
- Publication number
- CN117218089A CN117218089A CN202311200907.8A CN202311200907A CN117218089A CN 117218089 A CN117218089 A CN 117218089A CN 202311200907 A CN202311200907 A CN 202311200907A CN 117218089 A CN117218089 A CN 117218089A
- Authority
- CN
- China
- Prior art keywords
- depth
- depth map
- asphalt pavement
- absolute
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000010426 asphalt Substances 0.000 title claims abstract description 90
- 238000001514 detection method Methods 0.000 title abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 53
- 238000010276 construction Methods 0.000 claims abstract description 46
- 230000000007 visual effect Effects 0.000 claims abstract description 12
- 230000003287 optical effect Effects 0.000 claims abstract description 5
- 238000012937 correction Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- 238000013519 translation Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 11
- 238000004458 analytical method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 239000000463 material Substances 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 description 4
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 239000004576 sand Substances 0.000 description 4
- 238000010200 validation analysis Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 239000011384 asphalt concrete Substances 0.000 description 2
- 239000000306 component Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- QXJJQWWVWRCVQT-UHFFFAOYSA-K calcium;sodium;phosphate Chemical compound [Na+].[Ca+2].[O-]P([O-])([O-])=O QXJJQWWVWRCVQT-UHFFFAOYSA-K 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000845 maltitol Substances 0.000 description 1
- 239000013521 mastic Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/60—Planning or developing urban green infrastructure
Landscapes
- Road Repair (AREA)
Abstract
The application is applicable to the technical field of pavement detection, and provides a method for detecting the structural depth of an asphalt pavement, which comprises the following steps: acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement; inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of a reference image; acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value; fitting a depth map with an absolute depth value to obtain a fitting plane; correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement; and obtaining the construction depth of the asphalt pavement according to the accurate depth map. The application can improve the efficiency and the precision of detecting the construction depth of the asphalt pavement.
Description
Technical Field
The application belongs to the technical field of pavement detection, and particularly relates to a method for detecting the construction depth of an asphalt pavement.
Background
The surface structure and friction of asphalt pavement not only affect the driving comfort, but also affect the important factors of traffic safety when the vehicle accelerates and decelerates. The construction depth of the pavement in various national specifications is an important technical index of the skid resistance of the asphalt pavement, and the larger the construction depth of the pavement is, the better the skid resistance is. The construction depth of the newly built road surface is the largest, and with the increase of service life, the road surface can be gradually worn out, so that the friction between the wheels and the road surface is reduced. Traffic accidents caused by insufficient road friction are frequent. Therefore, the construction depth is an important detection index in newly built pavement acceptance and running pavement quality decay detection.
The common asphalt pavement skid resistance detection method mainly comprises a braking distance method, a pendulum instrument method, a sand paving method, a laser method and a transverse force coefficient method. Among them, the sanding method is a main method for detecting the depth of pavement structure, and is classified into an electric sanding method and a manual sanding method according to specifications of ASTM E965 and T0961-95, etc. The sanding method spreads fine sand in a cake shape on a road surface, records the geometric dimensions of the circle, and calculates the mean depth of construction (MTD, mean Texture Depth). The specification EN 13036-1 replaces fine sand with glass spheres, the latter having a smaller particle size range and being easier to push flat during operation. The sanding method has some disadvantages, such as great influence of the operation method on the result, time and labor waste. The fine sand is difficult to recycle, is not environment-friendly, cannot be used on a slope with a large angle, is greatly influenced by climate, and cannot be measured in rainy days or windy days. Nonetheless, the sanding method still serves as a basis for the accuracy of many new methods of measuring MTD.
The laser method is to measure the distance of the road surface by using line laser equipment, and draw an elevation scatter diagram of the section by using the distance value to reflect the concave-convex texture characteristic of the road surface. The correlation is calculated for the matching of the average section depth (MPD, mean Profile Depth) and the MTD obtained by the laser section analysis method, and the MTD is generally obtained by calculating the relative elevation or volume by the three-dimensional analysis method.
The road detection vehicle provided with the line laser scanning device can record microscopic texture fluctuation of the road surface during running, and some portable laser devices provide assistance for indoor and small-range detection. Cigada et al mount two industrial laser delta displacement sensors on a road detection vehicle to estimate road texture morphology at the estimated travel speed of the sensor, and a slower format speed, such as 10km/h, can measure wavelengths less than 0.5 mm. White et al developed a hand-held laser MTD device with relatively accurate ETD data over a sanding MTD range of 0.5mm to 1.5 mm. A ring texture tester (CTMeter) is also a mobile MTD detection device based on laser measurements. Abe et al and Hanson et al compare the MPD data obtained from CTMeter at the measurement point with the MTD, and obtain good correlation and smaller error results. The laser measurement is used for the fine texture analysis with high precision, but because invalid readings are easy to generate under unfavorable conditions of luminosity or light shadow, the triangulation has the problems of heavy sampling and filtering effects due to the influence of the external environment and the surface reflection characteristics of the pavement material.
In addition, zheng Mulian et al measured road surface skid resistance using SAFEGATE friction factor test vehicle. Wasilowska et al utilize anti-skid resistance tester and frictional force observer to test the skid resistance, and then evaluate road surface skid resistance. Dou Guangwu to evaluate the anti-skid property of the pavement, the depth of the pavement structure is measured by a high-precision laser ranging sensor. Zhou Xinglin and the like measure the construction depth of an asphalt pavement based on a laser vision method, and evaluate the pavement section depth (MPD) by an image processing method so as to evaluate the pavement anti-skid performance. Qian Zhendong and the like reconstruct a three-dimensional texture model of the rutting plate surface by using a digital image processing technology, calculate the fractal dimension of the three-dimensional texture model by a difference box dimension method, and research the relation between the fractal dimension and the anti-skid performance. Non-contact anti-skid resistances based on optical texture measurements by Ueckermann et al measure road texture to evaluate road skid resistance. The image-based automated system of Nejad et al evaluates road skid resistance by capturing road texture images via an automated Image Acquisition System (IAS). Liang et al calculate the Mean Texture Depth (MTD) based on a 3-D virtual model of the road surface texture generated from point cloud data in a three-dimensional detection system, and evaluate the road surface skid resistance. Cui and the like measure the average texture depth of the asphalt pavement based on multi-line laser and binocular vision technologies, and multi-line laser pairing and polar constraint technologies are introduced to realize image matching between multi-line laser and binocular vision, so that the average contour depth of the asphalt pavement is calculated in sequence.
In view of this, with the development of photographic equipment and three-dimensional reconstruction technology, the technology has been gradually developed for application research in analyzing surface texture and surface crack defects of asphalt concrete, and the like. Chen et al use a three-eye camera to collect road texture images and perform three-dimensional reconstruction, extract elevation data of the road, and calculate the MTD.The scanning device composed of a line laser and a double camera is developed by the et al, and horizontal and vertical correction of a scanning object is performed by using a camera image. In fact, both laser and structured light methods are active vision methods that emit active light sources, while the monocular, binocular and multi-view methods that are photographed by cameras are passive vision methods that utilize external light sources. However, these current methods of detecting the depth of a bituminous pavement structure are not efficient and accurate.
Disclosure of Invention
The embodiment of the application provides a method for detecting the construction depth of an asphalt pavement, which can solve the problem of poor efficiency and precision in detecting the construction depth of the asphalt pavement.
The embodiment of the application provides a method for detecting the construction depth of an asphalt pavement, which comprises the following steps:
acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement;
inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of a reference image;
acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value;
fitting a depth map with an absolute depth value to obtain a fitting plane;
correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement;
and obtaining the construction depth of the asphalt pavement according to the accurate depth map.
Optionally, obtaining the absolute depth value of the depth map includes:
four pairs of calibration points are obtained at four corners inside a calibration plate with known thickness; the four corners are in one-to-one correspondence with four pairs of calibration points;
obtaining an absolute depth value of the depth map through a depth value calculation formula;
the depth value calculation formula is:
wherein Z is abs Representing absolute depth values of depth map, Z rel Representing the relative depth value of the depth map, scale representing the scale factor of the calibration plate, x representing the thickness of the calibration plate, Z blue_i Representing the relative depth value of one of the ith pair of calibration points, Z red_i Representing the relative depth value of another index point in the ith pair of index points.
Optionally, clipping the depth map to obtain a depth map with an absolute depth value includes:
cutting the depth map by using a calibration plate to obtain a depth map with an absolute depth value; the size of the cut depth map is the same as that of the calibration plate.
Optionally, fitting the depth map with absolute depth values to obtain a fitting plane includes:
and fitting the depth map with the absolute depth value by using a RANSAC algorithm to obtain a fitting plane.
Optionally, the obtaining an accurate depth map of the asphalt pavement based on correction of the inclination error of the fitting plane includes:
obtaining a normal vector n of a fitting plane;
obtaining an accurate depth map of the asphalt pavement through an inclination correction formula;
the inclination correction formula is:
wherein Z' represents an accurate depth map of the asphalt pavement, T represents a coordinate transformation matrix, X, Y represents pixel coordinates in the depth map, Z represents an absolute depth value of the depth map, R represents a rotation matrix, θ represents the angle between normal vector n and Z axis, +.>t represents a three-dimensional translation vector->Representing a 3 x 1 transpose of the zero vector.
Optionally, obtaining the construction depth of the asphalt pavement according to the accurate depth map includes:
by the formulaCalculating to obtain the construction depth of the asphalt pavement;
wherein the MTD p Representing the construction depth of the asphalt pavement, M and N respectively represent the pixel number in the length and width directions of the accurate depth map, Z mn Representing absolute depth values of nth pixels in mth row and nth column of accurate depth map, Z p Representing the absolute depth value of the selected texture reference surface,y represents the area within the calibration plate.
Optionally, obtaining the construction depth of the asphalt pavement according to the accurate depth map includes:
by the formulaCalculating to obtain the construction depth of the asphalt pavement;
wherein, MPD p Represents the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth,depth value representing average elevation of jth row of pixels in accurate depth map, +.>And the depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
Alternatively, the resolution of the images of the multiple views is 3024×3024, and n has a value of 6.
The scheme of the application has the following beneficial effects:
in the embodiment of the application, a reference image of an asphalt pavement and n source images shot around the asphalt pavement are input into a depth map reconstruction model to reconstruct a depth map, the depth map of the reference image is obtained, the depth map is cut by acquiring the absolute depth value of the depth map, an effective area of the depth map is obtained, then the cut depth map is fitted, and the accurate depth map of the asphalt pavement is obtained by correcting the inclination error based on a fitting plane, so that the construction depth of the asphalt pavement is obtained based on the accurate depth map. The depth map is obtained by combining a computer vision technology and a deep learning technology, so that the efficiency and the precision of reconstructing the depth map can be greatly enhanced, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be improved, and meanwhile, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be greatly improved because the detection of the structural depth is based on an effective area of the depth map and is completed based on the depth map after inclination error correction.
In addition, the detection method integrates automatic image acquisition and analysis, so the detection method is a truly complete automatic detection method, is applicable to images of various asphalt pavements, and has extremely strong practical significance.
Other advantageous effects of the present application will be described in detail in the detailed description section which follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for detecting the depth of an asphalt pavement structure according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a calibration plate according to an embodiment of the present application;
FIG. 3 is a graph showing the variation of depth values of a section before and after the tilt error processing according to an embodiment of the present application;
FIG. 4 is a diagram illustrating MAPE statistics corresponding to different texture reference planes when calculating the depth of the texture according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an MPD according to an embodiment of the present application p Is a schematic diagram of the calculation process.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Aiming at the problem of poor efficiency and precision of detecting the construction depth of the asphalt pavement at present, the embodiment of the application provides a method for detecting the construction depth of the asphalt pavement, which comprises the steps of inputting a reference image of the asphalt pavement and n source images shot around the asphalt pavement into a depth map reconstruction model to reconstruct the depth map to obtain the depth map of the reference image, clipping the depth map to obtain an effective area of the depth map by acquiring an absolute depth value of the depth map, fitting the clipped depth map, and correcting an inclination error based on a fitting plane to obtain an accurate depth map of the asphalt pavement so as to obtain the construction depth of the asphalt pavement based on the accurate depth map.
The depth map is obtained by combining a computer vision technology and a deep learning technology, so that the efficiency and the precision of reconstructing the depth map can be greatly enhanced, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be improved, and meanwhile, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be greatly improved because the detection of the structural depth is based on an effective area of the depth map and is completed based on the depth map after inclination error correction.
The method for detecting the depth of the asphalt pavement structure provided by the embodiment of the application is exemplified by the following specific examples.
As shown in fig. 1, the method for detecting the depth of the asphalt pavement structure provided by the embodiment of the application comprises the following steps:
and 11, acquiring images of the asphalt pavement at a plurality of view angles.
The images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement, the images of the multiple visual angles are all RGB images, and RGB represents the colors of three channels of red, green and blue.
The resolution and the number of the images with the plurality of view angles can be properly adjusted according to the performance of the camera. As a preferred example, the resolution of the images of the plurality of view angles may be 3024×3024, the n may be 6, 7, 8, 9, 10, etc., and the camera may be a digital camera such as a camera of a smart phone.
And step 12, inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of the reference image.
The n source images are mainly used for assisting in reconstructing a depth map of the reference image.
In some embodiments of the present application, the depth map reconstruction model may be a conventional depth map reconstruction model. As a preferred example, patchmatchNet (PatchmatchNet is a deep learning model for image processing) may be used.
The depth map reconstruction model in the above step 12 is a trained model. The training process of the model is described here by taking PatchmatchNet as an example.
Firstly, acquiring image data by adopting a depth camera Intel RealSense D405 to obtain an RGB-D (RGB-D=common RGB three-channel color image+depth image) training set (covering a common dense graded asphalt concrete mixture (AC), asphalt mastic macadam mixture (SMA), upgrading asphalt mixture pavement such as a water distribution type wearing layer mixture (OGFC) and the like), and carrying out preprocessing operations such as finishing, cutting and the like on the RGB-D training set; the model is then pre-trained using a large-scale public dataset (e.g., DTU dataset), the weight of the pre-trained model is used as an initialization parameter, and then the model is fine-tuned using the pre-processed RGB-D training set.
It is worth mentioning that, by training the model by the above-mentioned training method of transfer learning, the feature and parameter initialization value learned by the pre-training model can be utilized to accelerate the training of the model and improve the performance, and reduce the data amount requirement and training time. The model further learns road texture features from the road texture RGB-D dataset based on the pre-training weights, and the model finally achieves good reconstruction performance.
After model training is finished, ioU index (IoU index is a standard for measuring accuracy of detecting corresponding objects in a specific data set) is optionally used for evaluating accuracy of neural network training, and point cloud reconstructed by the model is compared with true point cloud.
The accuracy of the model can be evaluated in particular on an RGB validation set (available for capture with a digital camera). Is defined asWherein V is GT∩Pre Representing the intersection of the true value point cloud and the point cloud reconstructed by the model, V GT∪Pre And representing the union set of the true value point cloud and the point cloud reconstructed by the model, wherein the true value point cloud can be obtained by inputting an RGB verification set image into PhotoScan software for reconstruction.
The IoU index is in a value range of 0 to 1, when the value of IoU is closer to 1, the spatial morphology of the 2 three-dimensional models is closer, namely, the accuracy of the PatchmatchNet prediction point cloud is better, the model can be deduced to reconstruct a depth map with good quality.
In some embodiments of the present application, to improve the accuracy of the PatchmatchNet reconstructed depth map, after model training and verification is completed, the effects of different image resolutions and view angle numbers, as well as pavement material types, on the reconstructed quality can also be analyzed.
To ensure that the dataset contains diverse scenes and conditions to obtain a comprehensive analysis result, the analysis can be performed on an RGB validation set containing different image resolutions, number of views, and pavement material types. In a specific analysis procedure, different image resolutions (2048, 3024, 3400) and viewing angle numbers (5, 7, 10) were used for combination and testing. The image resolution (3024) and the number of views (7) are then fixed and tested on different pavement types and the corresponding reconstruction results and evaluation index (IoU index may be used) results recorded.
And comparing the influence of different factors on the reconstruction quality by carrying out statistical analysis on the experimental result. The conclusion is that the reconstruction quality is best under the condition that the image resolution is 3024 multiplied by 3024 and the view angle is 7, the reconstruction quality on different pavement material types is not greatly different, and the effect is stable, so that the reconstruction task in practical application can obtain good effect.
In some embodiments of the application, to demonstrate the accuracy of the PatchmatchNet of the application, an RGB validation set was selected as the test subject (specifically screening images with image resolution 3024 x 3024, view angle number 7, and different pavement material types); selecting a traditional three-dimensional reconstruction model colmap as a reference model, using the model to carry out three-dimensional reconstruction on the image in the data set, and obtaining a corresponding reconstruction result; the IoU index is used to measure the quality of point cloud reconstruction under different models.
For the selected RGB validation set, the PatchmatchNet and the traditional three-dimensional reconstruction model were used for reconstruction, respectively. The reconstruction results obtained by the two methods are compared, including visual reconstruction depth map comparison and quantitative IoU index comparison.
The two reconstruction results are quantitatively evaluated using the selected evaluation index. The patch matchnet is better than the conventional three-dimensional reconstruction model in terms of reconstruction quality IoU =0.77 for the point cloud, ioU =0.64, and far better in terms of reconstruction efficiency than the conventional three-dimensional reconstruction model.
And step 13, obtaining an absolute depth value of the depth map, and cutting the depth map to obtain the depth map with the absolute depth value.
In some embodiments of the present application, the absolute depth value of the depth map may be determined using calibration plates of known thickness and cropped to obtain a depth map having the absolute depth value.
And step 14, fitting the depth map with the absolute depth value to obtain a fitting plane.
In some embodiments of the present application, a random consensus sampling (RANSAC) algorithm may be used to fit a depth map with absolute depth values to obtain a fit plane.
And 15, correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement.
In some embodiments of the present application, the accurate depth map of the asphalt pavement may be obtained by obtaining the normal vector n of the fitting plane and then projecting the points in the depth map into the rotated coordinate system through the coordinate transformation matrix T.
And step 16, obtaining the construction depth of the asphalt pavement according to the accurate depth map.
In some embodiments of the application, the depth of construction of the asphalt pavement may be obtained based on a three-dimensional index of volume or a two-dimensional index of profile.
It is worth mentioning that, because the depth map is obtained by combining the computer vision technology and the deep learning technology, the efficiency and the precision of the depth map reconstruction can be greatly enhanced, so that the efficiency and the precision of the detection of the construction depth of the asphalt pavement can be improved, and meanwhile, because the detection of the construction depth is based on the effective area of the depth map and is completed based on the depth map after the inclination error correction, the efficiency and the precision of the detection of the construction depth of the asphalt pavement can be greatly improved.
The specific implementation manner of obtaining the absolute depth value of the depth map and clipping the depth map to obtain the depth map with the absolute depth value is illustrated in the following step 13 with reference to the specific embodiment.
In some embodiments of the present application, four pairs of calibration points may be obtained at four corners inside a calibration plate of known thickness, and then the absolute depth value of the depth map may be obtained by a depth value calculation formula, and the depth map may be cut by using the calibration plate to obtain a depth map having the absolute depth value.
The four corners are in one-to-one correspondence with the four pairs of calibration points, namely, each corner is provided with a pair of calibration points, and the calibration points of the 4 corners are arranged in the same way. To facilitate understanding of the calibration points, the calibration points are illustrated herein by way of example with the calibration plate shown in FIG. 2. Fig. 2 illustrates only a pair of calibration points for one of the four corners, including a circular calibration point on the upper side of the corner and a triangular calibration point on the lower side.
The depth value calculation formula is as follows:
wherein Z is abs Representing absolute depth values of the depth map; z is Z rel Representing relative depth values of the depth map; scale represents the scale factor of the calibration plate,x represents the thickness of the calibration plate, and can be specifically set according to practical values, such as 3 mm; z is Z blue_i Representing the relative depth of one of the ith pair of calibration points (e.g., the circular calibration point in fig. 2)A degree value; z is Z red_i Representing the relative depth value of the other index point of the ith pair of index points (e.g., the triangle index point in fig. 2).
It should be noted that, the depth map reconstructed by PatchmatchNet can reflect the relative depth value, Z blue_i And Z red_i The relative depth value in the depth map obtained by reconstructing the depth map of the calibration plate by using PatchmatchNet.
In some embodiments of the present application, the depth map is cropped using a calibration plate in order to screen out the effective area (also referred to as the range of interest) of the depth map. The size of the depth map after clipping is the same as the size of the calibration plate. As a preferred example, the dimensions of the cropped depth map may be 100mm x 100mm.
The specific implementation of the step 15 to obtain an accurate depth map of the asphalt pavement based on correction of the inclination error of the fitting plane is exemplarily described below with reference to a specific embodiment.
In some embodiments of the present application, the normal vector n of the fitting plane may be obtained first, and then the accurate depth map of the asphalt pavement may be obtained by the inclination correction formula
The inclination correction formula is as follows:
wherein Z' represents an accurate depth map of the asphalt pavement; t represents a coordinate transformation matrix for describing a matrix form of a transformation relationship from one coordinate system to another coordinate system, which is set according to the selected coordinate system and transformation rule,x and Y represent pixel point coordinates in the depth map; z represents the absolute depth value of the depth map; r represents a rotation matrix, ">θ represents a normal vectorn and Z axis, the Z axis is the Z axis of a camera (particularly, the camera which can acquire images of a plurality of visual angles) coordinate system>t denotes a three-dimensional translation vector, which can be used to describe the translation transformation from one coordinate system to another, which will be translated accordingly on each coordinate axis when adding one vector to the translation vector; />Representing a 3 x 1 transpose of the zero vector.
It is worth mentioning that correction of inclination error can effectively correct the inclination error that exists on the road surface, promotes the precision of structure degree of depth. As shown in fig. 3, the absolute difference of the depth values after the inclination error correction becomes smaller, and the overall slope of the cross-sectional depth values becomes smaller, which means that the inclination error of the road surface is effectively corrected. Where the abscissa in fig. 3 represents pixel values of a certain row of the depth map and the ordinate represents depth values relative to the texture reference plane.
The specific implementation of step 16 to obtain the construction depth of the asphalt pavement according to the accurate depth map is exemplarily described below with reference to specific embodiments.
The process for obtaining the construction depth of the asphalt pavement based on the three-dimensional index of the volume comprises the following steps:
by the formulaAnd calculating to obtain the construction depth of the asphalt pavement.
Wherein the MTD p Representing the construction depth of the asphalt pavement, M and N respectively represent the pixel number in the length and width directions of the accurate depth map, Z mn Representing absolute depth values of nth pixels in mth row and nth column of accurate depth map, Z p Representing the absolute depth value of the selected texture reference surface,y represents the area within the calibration plate. It should be noted that, a plane where the p% minimum depth quantile value in the accurate depth map is located may be selected as the texture reference plane.
In some embodiments of the present application, a plane in which a depth quantile value at 5% of an accurate depth map is located may be selected as a texture reference plane, where prediction errors of various types of road surfaces are close to minimum. It will be appreciated that in embodiments of the application, it is not limited what plane of the texture reference surface is specifically the depth map.
To facilitate clarity of the effect of texture references on the texture depth, different texture references are selected to calculate the texture depth, and MAPE statistics are used to measure the average relative error percentage of the texture depth to the measured value.
Wherein the MTD e Indicating the measured value.
As shown in FIG. 4, tests were performed on 4 different types of asphalt pavements, AC-13, AC-16, SMA-13, OGFC-16, to obtain MAPE statistical indexes corresponding to the construction depths calculated by taking planes at different depth quantile values as texture reference planes. In fig. 4, the abscissa indicates the value of p, and the ordinate indicates the MAPE statistical index value.
The process for obtaining the construction depth of the asphalt pavement based on the two-dimensional index of the section comprises the following steps:
by the formulaAnd calculating to obtain the construction depth of the asphalt pavement.
Wherein, MPD p Represents the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth,depth value representing average elevation of jth row of pixels in accurate depth map, +.>And the depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
The depth value of the average elevation and the depth value of the peak elevation may be obtained from the MPD shown in FIG. 5 p Is selected from the calculation process schematic diagram of (a). Where the abscissa of fig. 5 represents the pixel values of a certain row of the depth map, the ordinate represents the depth values relative to the texture reference plane, line C represents the peak elevation plane, and line D represents the average elevation plane.
It should be further noted that, in the practical application process, the three-dimensional index based on volume or the two-dimensional index based on section can be selected according to the practical situation to obtain the construction depth of the asphalt pavement.
An exemplary description will be given below of a related apparatus that implements the detection method provided in the present embodiment.
The equipment core components comprise a wide-angle macro lens, a single-lens reflex, a rotary servo motor, a horizontal sliding module, a fixed bracket and the like; the shooting angle of the camera is controlled by the servo motor and the sliding module, and the shooting quantity of the measuring points is 1 (the reference image is shot perpendicular to the road surface) +8 (multi-view image) and is 9 in total. The camera is directly connected with the calculation server, and the photo is exported in real time to conduct pavement structure depth prediction; the light shield is required to be designed, and external light sources such as natural light are replaced by a form of internally provided light sources, so that the influence on imaging results caused by uneven or serious inclination of the ambient light sources is avoided; the equipment is portable and light in operation, and the gravity center deviation of the equipment is avoided.
In summary, the asphalt pavement structure depth detection method provided by the embodiment of the application has the following advantages:
first, non-contact: the non-contact measuring method is a non-contact measuring method, does not need to physically contact the measured object like a sanding method, and is more convenient and safer;
secondly, enriching information: rich additional information may be provided. In addition to constructing depth values, the depth image may also provide other information about the constructed surface, such as texture, color, etc. Such information may be used for further analysis and applications such as road surface quality assessment, defect detection, etc.
Third, visual display: visual depth images can be generated, and distribution and change conditions of the construction depth values can be intuitively displayed. This facilitates the understanding and analysis of the depth of construction by researchers and decision makers, and supports subsequent decisions and processing.
While the foregoing is directed to the preferred embodiments of the present application, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.
Claims (8)
1. A method for detecting the depth of an asphalt pavement structure, comprising the steps of:
acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement;
inputting the images of the multiple view angles into a depth map reconstruction model to reconstruct a depth map, and obtaining a depth map of the reference image;
acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value;
fitting the depth map with the absolute depth value to obtain a fitting plane;
correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement;
and obtaining the construction depth of the asphalt pavement according to the accurate depth map.
2. The method of claim 1, wherein the obtaining absolute depth values of the depth map comprises:
four pairs of calibration points are obtained at four corners inside a calibration plate with known thickness; the four corners are in one-to-one correspondence with the four pairs of calibration points;
obtaining an absolute depth value of the depth map through a depth value calculation formula;
the depth value calculation formula is as follows:
wherein Z is abs Representing absolute depth values of the depth map, Z rel Representing the relative depth value of the depth map, scale representing the scale factor of the calibration plate, x representing the thickness of the calibration plate, Z blue_i Representing the relative depth value of one of the ith pair of calibration points, Z red_i Representing the relative depth value of another index point in the ith pair of index points.
3. The method of claim 2, wherein cropping the depth map to obtain a depth map having the absolute depth value comprises:
cutting the depth map by using the calibration plate to obtain a depth map with the absolute depth value; the size of the cut depth map is the same as that of the calibration plate.
4. The method of claim 1, wherein fitting the depth map having the absolute depth values to obtain a fit plane comprises:
and fitting the depth map with the absolute depth value by using a RANSAC algorithm to obtain a fitting plane.
5. The method of claim 2, wherein the correcting for tilt errors based on the fitted plane obtains an accurate depth map of the asphalt pavement, comprising:
obtaining a normal vector n of the fitting plane;
obtaining an accurate depth map of the asphalt pavement through an inclination correction formula;
the inclination correction formula is as follows:
wherein Z' represents an accurate depth map of the asphalt pavement, T represents a coordinate transformation matrix, X, Y represents pixel coordinates in the depth map, Z represents an absolute depth value of the depth map, R represents a rotation matrix,θ represents the angle between normal vector n and Z axis, +.> t represents a three-dimensional translation vector->Representing a 3 x 1 transpose of the zero vector.
6. The method of claim 5, wherein the obtaining the construction depth of the asphalt pavement from the accurate depth map comprises:
by the formulaCalculating to obtain the construction depth of the asphalt pavement;
wherein the MTD p Representing the construction depth of the asphalt pavement, M and N respectively representing the pixel number in the length and width directions of the accurate depth map, Z mn Representing absolute depth values of nth pixels in an mth row and nth pixels in the accurate depth map, Z p Representation ofThe absolute depth value of the selected texture reference surface,u represents the area within the calibration plate.
7. The method of claim 5, wherein the obtaining the construction depth of the asphalt pavement from the accurate depth map comprises:
by the formulaCalculating to obtain the construction depth of the asphalt pavement;
wherein, MPD p Representing the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth,depth value representing the average elevation of the j-th row of pixels in said exact depth map,/->And a depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
8. The method of claim 1, wherein the resolution of the images at the plurality of view angles is 3024 x 3024 and n has a value of 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311200907.8A CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311200907.8A CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218089A true CN117218089A (en) | 2023-12-12 |
| CN117218089B CN117218089B (en) | 2024-04-19 |
Family
ID=89043833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311200907.8A Active CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218089B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
| CN112489202A (en) * | 2020-12-08 | 2021-03-12 | 甘肃智通科技工程检测咨询有限公司 | Pavement macroscopic texture reconstruction method based on multi-view deep learning |
| CN112927366A (en) * | 2021-05-10 | 2021-06-08 | 中南大学 | Asphalt pavement structure depth measuring method |
| WO2021115071A1 (en) * | 2019-12-12 | 2021-06-17 | 中国科学院深圳先进技术研究院 | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device |
| CN113066168A (en) * | 2021-04-08 | 2021-07-02 | 云南大学 | Multi-view stereo network three-dimensional reconstruction method and system |
| CN113962858A (en) * | 2021-10-22 | 2022-01-21 | 沈阳工业大学 | Multi-view depth acquisition method |
| CN114280599A (en) * | 2021-11-15 | 2022-04-05 | 江苏金晓电子信息股份有限公司 | Coordinate conversion matching vehicle detection method based on millimeter wave radar and video data |
| CN114812450A (en) * | 2022-04-25 | 2022-07-29 | 山东省路桥集团有限公司 | Machine vision-based asphalt pavement construction uniformity detection and evaluation method |
| CN114994054A (en) * | 2022-05-26 | 2022-09-02 | 哈尔滨工业大学 | Method for determining relation between road surface texture average construction depth and average section depth |
-
2023
- 2023-09-18 CN CN202311200907.8A patent/CN117218089B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
| WO2021115071A1 (en) * | 2019-12-12 | 2021-06-17 | 中国科学院深圳先进技术研究院 | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device |
| CN112489202A (en) * | 2020-12-08 | 2021-03-12 | 甘肃智通科技工程检测咨询有限公司 | Pavement macroscopic texture reconstruction method based on multi-view deep learning |
| CN113066168A (en) * | 2021-04-08 | 2021-07-02 | 云南大学 | Multi-view stereo network three-dimensional reconstruction method and system |
| CN112927366A (en) * | 2021-05-10 | 2021-06-08 | 中南大学 | Asphalt pavement structure depth measuring method |
| CN113962858A (en) * | 2021-10-22 | 2022-01-21 | 沈阳工业大学 | Multi-view depth acquisition method |
| CN114280599A (en) * | 2021-11-15 | 2022-04-05 | 江苏金晓电子信息股份有限公司 | Coordinate conversion matching vehicle detection method based on millimeter wave radar and video data |
| CN114812450A (en) * | 2022-04-25 | 2022-07-29 | 山东省路桥集团有限公司 | Machine vision-based asphalt pavement construction uniformity detection and evaluation method |
| CN114994054A (en) * | 2022-05-26 | 2022-09-02 | 哈尔滨工业大学 | Method for determining relation between road surface texture average construction depth and average section depth |
Non-Patent Citations (4)
| Title |
|---|
| HAN-CHENG DAN 等: "An improved computation method for asphalt pavement texture depth based on multicular vision 3D reconstruction technology", 《CONSTRUCTION AND BUILDING MATERIALS》, 18 January 2022 (2022-01-18), pages 1 - 12 * |
| HAN-CHENG DAN 等: "Evaluation of asphalt pavement texture using multiview stereo reconstruction based on deep learning", 《CONSTRUCTION AND BUILDING MATERIALS》, 5 January 2024 (2024-01-05), pages 1 - 17 * |
| 陈钰闻: "沥青路面离析检测算法研究", 《中国优秀硕士学位论文全文数据库工程科技II辑》, 15 June 2023 (2023-06-15), pages 034 - 53 * |
| 陈鹏: "基于（图像）三维数据的沥青路面构造深度算法研究", 《中国优秀硕士学位论文全文数据库工程科技II辑》, 15 March 2015 (2015-03-15), pages 034 - 329 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117218089B (en) | 2024-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322702B (en) | Intelligent vehicle speed measuring method based on binocular stereo vision system | |
| CN112927366B (en) | Asphalt pavement structure depth measuring method | |
| CN101354241B (en) | Method for evaluating aggregate digital image | |
| CN106978774B (en) | A kind of road surface pit slot automatic testing method | |
| CN107179322A (en) | A kind of bridge bottom crack detection method based on binocular vision | |
| CN108107462A (en) | The traffic sign bar gesture monitoring device and method that RTK is combined with high speed camera | |
| Dan et al. | An improved computation method for asphalt pavement texture depth based on multiocular vision 3D reconstruction technology | |
| Liu et al. | The detection effect of pavement 3D texture morphology using improved binocular reconstruction algorithm with laser line constraint | |
| CN111473776A (en) | Landslide crack monitoring method based on single-image close-range photogrammetry | |
| CN112033800B (en) | Method for measuring overall volume change of sandy soil triaxial sample | |
| CN111179335A (en) | Standing tree measuring method based on binocular vision | |
| CN114972177A (en) | Road disease identification management method and device and intelligent terminal | |
| Medeiros Jr et al. | 3D pavement macrotexture parameters from close range photogrammetry | |
| CN116342674A (en) | Method for calculating asphalt pavement construction depth by three-dimensional model | |
| CN117437368B (en) | Unmanned plane-based pavement evenness measuring method, system, terminal and medium | |
| Motayyeb et al. | Fusion of UAV-based infrared and visible images for thermal leakage map generation of building facades | |
| CN117218089B (en) | Asphalt pavement structure depth detection method | |
| Li et al. | Road Damage Evaluation via Stereo Camera and Deep Learning Neural Network | |
| Guan et al. | Multi-scale asphalt pavement deformation detection and measurement based on machine learning of full field-of-view digital surface data | |
| CN105046703A (en) | Image analysis atmospheric visibility method | |
| CN114283121A (en) | Paving width detection method and device, readable storage medium and paver | |
| CN111429430B (en) | Tunnel detection vehicle size mapping relation calculation method based on machine vision | |
| Chou et al. | Road marking retroreflectivity study via a visual algorithm | |
| CN112907739B (en) | Method, device and system for acquiring height difference information of well lid | |
| Jahanshahi et al. | A novel system for road surface monitoring using an inexpensive infrared laser sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |