CN117218089B - Asphalt pavement structure depth detection method - Google Patents
Asphalt pavement structure depth detection method Download PDFInfo
- Publication number
- CN117218089B CN117218089B CN202311200907.8A CN202311200907A CN117218089B CN 117218089 B CN117218089 B CN 117218089B CN 202311200907 A CN202311200907 A CN 202311200907A CN 117218089 B CN117218089 B CN 117218089B
- Authority
- CN
- China
- Prior art keywords
- depth
- depth map
- asphalt pavement
- absolute
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000010426 asphalt Substances 0.000 title claims abstract description 89
- 238000001514 detection method Methods 0.000 title abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000010276 construction Methods 0.000 claims abstract description 46
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 230000003287 optical effect Effects 0.000 claims abstract description 5
- 238000012937 correction Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- 238000013519 translation Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 11
- 238000004458 analytical method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 description 4
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 239000004576 sand Substances 0.000 description 4
- 238000010200 validation analysis Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 239000011384 asphalt concrete Substances 0.000 description 2
- 239000000306 component Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- QXJJQWWVWRCVQT-UHFFFAOYSA-K calcium;sodium;phosphate Chemical compound [Na+].[Ca+2].[O-]P([O-])([O-])=O QXJJQWWVWRCVQT-UHFFFAOYSA-K 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000845 maltitol Substances 0.000 description 1
- 239000013521 mastic Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/60—Planning or developing urban green infrastructure
Abstract
The application is applicable to the technical field of pavement detection, and provides a method for detecting the structural depth of an asphalt pavement, which comprises the following steps: acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement; inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of a reference image; acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value; fitting a depth map with an absolute depth value to obtain a fitting plane; correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement; and obtaining the construction depth of the asphalt pavement according to the accurate depth map. The application can improve the efficiency and the precision of detecting the construction depth of the asphalt pavement.
Description
Technical Field
The application belongs to the technical field of pavement detection, and particularly relates to a method for detecting the construction depth of an asphalt pavement.
Background
The surface structure and friction of asphalt pavement not only affect the driving comfort, but also affect the important factors of traffic safety when the vehicle accelerates and decelerates. The construction depth of the pavement in various national specifications is an important technical index of the skid resistance of the asphalt pavement, and the larger the construction depth of the pavement is, the better the skid resistance is. The construction depth of the newly built road surface is the largest, and with the increase of service life, the road surface can be gradually worn out, so that the friction between the wheels and the road surface is reduced. Traffic accidents caused by insufficient road friction are frequent. Therefore, the construction depth is an important detection index in newly built pavement acceptance and running pavement quality decay detection.
The common asphalt pavement skid resistance detection method mainly comprises a braking distance method, a pendulum instrument method, a sand paving method, a laser method and a transverse force coefficient method. Among them, the sanding method is a main method for detecting the depth of pavement structure, and is classified into an electric sanding method and a manual sanding method according to specifications of ASTM E965 and T0961-95, etc. The sanding method spreads fine sand in a cake shape on a road surface, records the geometric dimensions of the circle, and calculates the mean depth of construction (MTD, mean Texture Depth). Specification EN 13036-1 replaces fine sand with glass spheres, which have a smaller particle size range and are easier to push flat during operation. The sanding method has some disadvantages, such as great influence of the operation method on the result, time and labor waste. The fine sand is difficult to recycle, is not environment-friendly, cannot be used on a slope with a large angle, is greatly influenced by climate, and cannot be measured in rainy days or windy days. Nonetheless, the sanding method still serves as a basis for the accuracy of many new methods of measuring MTD.
The laser method is to measure the distance of the road surface by using line laser equipment, and draw an elevation scatter diagram of the section by using the distance value to reflect the concave-convex texture characteristic of the road surface. The correlation needs to be calculated for the matching of the average section depth (MPD, mean Profile Depth) and the MTD by the laser section analysis method, and the MTD is generally obtained by the three-dimensional analysis method through the calculation of the relative elevation or volume.
The road detection vehicle provided with the line laser scanning device can record microscopic texture fluctuation of the road surface during running, and some portable laser devices provide assistance for indoor and small-range detection. Cigada et al mount two industrial laser delta displacement sensors on a road detection vehicle to estimate road texture morphology with the estimated travel speed of the sensor, and slower form speeds such as 10km/h can measure wavelengths less than 0.5 mm. White et al developed a hand-held laser MTD device with relatively accurate ETD data over a sanding MTD range of 0.5mm to 1.5 mm. The ring texture tester (CTMeter) is also a mobile MTD detection device based on laser measurements. Abe et al and Hanson et al compare CTMeter measured point MPD data with MTD, resulting in good correlation and smaller error results. The laser measurement is used for the fine texture analysis with high precision, but because invalid readings are easy to generate under unfavorable conditions of luminosity or light shadow, the triangulation has the problems of heavy sampling and filtering effects due to the influence of the external environment and the surface reflection characteristics of the pavement material.
In addition, zheng Mulian et al used SAFEGATE friction factor test vehicle to measure road surface skid resistance. WASILEWSKA and the like test the anti-skid force by using an anti-skid resistance tester and a friction force observer, so as to evaluate the anti-skid performance of the pavement. Dou Guangwu is to evaluate the anti-skid performance of the pavement, and to measure the pavement construction depth by using a high-precision laser ranging sensor. Zhou Xinglin and the like measure the construction depth of the asphalt pavement based on a laser vision method, and estimate the pavement section depth (MPD) by an image processing method so as to evaluate the skid resistance of the pavement. Qian Zhendong reconstructing a rutting plate surface three-dimensional texture model by using a digital image processing technology, calculating the fractal dimension of the three-dimensional texture model by using a difference box dimension method, and researching the relation between the fractal dimension and the anti-skid performance. Ueckermann and the like, which are based on optical texture measurement, measure the road texture to evaluate the road anti-skid performance. Nejad et al image-based automated systems evaluate road surface skid resistance by capturing road surface texture images via an automated Image Acquisition System (IAS). Liang et al calculate the Mean Texture Depth (MTD) based on a 3-D virtual model of the road surface texture generated from point cloud data in a three-dimensional detection system, and evaluate the road surface skid resistance. Cui and the like measure the average texture depth of the asphalt pavement based on multi-line laser and binocular vision technologies, and multi-line laser pairing and polar constraint technologies are introduced to realize image matching between multi-line laser and binocular vision, so that the average contour depth of the asphalt pavement is calculated in sequence.
In view of this, with the development of photographic equipment and three-dimensional reconstruction technology, the technology has been gradually developed for application research in analyzing surface texture and surface crack defects of asphalt concrete, and the like. Chen et al use a three-eye camera to collect road texture images and perform three-dimensional reconstruction, extract elevation data of the road, and calculate the MTD.The scanning device composed of a line laser and a double camera is developed by the et al, and horizontal and vertical correction of a scanning object is performed by using a camera image. In fact, both laser and structured light methods are active vision methods that emit active light sources, while the monocular, binocular and multi-view methods that are photographed by cameras are passive vision methods that utilize external light sources. However, these current methods of detecting the depth of a bituminous pavement structure are not efficient and accurate.
Disclosure of Invention
The embodiment of the application provides a method for detecting the construction depth of an asphalt pavement, which can solve the problem of poor efficiency and precision in detecting the construction depth of the asphalt pavement.
The embodiment of the application provides a method for detecting the construction depth of an asphalt pavement, which comprises the following steps:
Acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement;
Inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of a reference image;
Acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value;
Fitting a depth map with an absolute depth value to obtain a fitting plane;
Correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement;
and obtaining the construction depth of the asphalt pavement according to the accurate depth map.
Optionally, obtaining the absolute depth value of the depth map includes:
Four pairs of calibration points are obtained at four corners inside a calibration plate with known thickness; the four corners are in one-to-one correspondence with four pairs of calibration points;
Obtaining an absolute depth value of the depth map through a depth value calculation formula;
The depth value calculation formula is:
Wherein Z abs represents the absolute depth value of the depth map, Z rel represents the relative depth value of the depth map, scale represents the scale factor of the calibration plate, x represents the thickness of the calibration plate, Z blue_i represents the relative depth value of one of the ith pair of calibration points, and Z red_i represents the relative depth value of the other of the ith pair of calibration points.
Optionally, clipping the depth map to obtain a depth map with an absolute depth value includes:
Cutting the depth map by using a calibration plate to obtain a depth map with an absolute depth value; the size of the cut depth map is the same as that of the calibration plate.
Optionally, fitting the depth map with absolute depth values to obtain a fitting plane includes:
And fitting the depth map with the absolute depth value by using a RANSAC algorithm to obtain a fitting plane.
Optionally, the obtaining an accurate depth map of the asphalt pavement based on correction of the inclination error of the fitting plane includes:
Obtaining a normal vector n of a fitting plane;
obtaining an accurate depth map of the asphalt pavement through an inclination correction formula;
The inclination correction formula is:
Wherein Z' represents an accurate depth map of the asphalt pavement, T represents a coordinate transformation matrix, X, Y represents pixel coordinates in the depth map, Z represents an absolute depth value of the depth map, R represents a rotation matrix, Θ represents the angle between normal vector n and Z axis,/>T represents a three-dimensional translation vector,/>Representing a 3 x1 transpose of the zero vector.
Optionally, obtaining the construction depth of the asphalt pavement according to the accurate depth map includes:
By the formula Calculating to obtain the construction depth of the asphalt pavement;
Wherein MTD p represents the construction depth of the asphalt pavement, M and N represent the pixel numbers of the length and width directions of the accurate depth map respectively, Z mn represents the absolute depth value of the nth pixel of the M-th row in the accurate depth map, Z p represents the absolute depth value of the selected texture reference surface, Y represents the area within the calibration plate.
Optionally, obtaining the construction depth of the asphalt pavement according to the accurate depth map includes:
By the formula Calculating to obtain the construction depth of the asphalt pavement;
Wherein MPD p represents the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth, Depth value representing average elevation of jth row of pixels in accurate depth map,/>And the depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
Alternatively, the resolution of the images of the multiple views is 3024×3024, and n has a value of 6.
The scheme of the application has the following beneficial effects:
In the embodiment of the application, a reference image of an asphalt pavement and n source images shot around the asphalt pavement are input into a depth map reconstruction model to reconstruct a depth map, the depth map of the reference image is obtained, the depth map is cut by acquiring the absolute depth value of the depth map, an effective area of the depth map is obtained, then the cut depth map is fitted, and the accurate depth map of the asphalt pavement is obtained by correcting the inclination error based on a fitting plane, so that the construction depth of the asphalt pavement is obtained based on the accurate depth map. The depth map is obtained by combining a computer vision technology and a depth learning technology, so that the efficiency and the precision of reconstructing the depth map can be greatly enhanced, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be improved, and meanwhile, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be greatly improved because the detection of the structural depth is based on an effective area of the depth map and is completed based on the depth map after inclination error correction.
In addition, the detection method integrates automatic image acquisition and analysis, so the detection method is a truly complete automatic detection method, is applicable to images of various asphalt pavements, and has extremely strong practical significance.
Other advantageous effects of the present application will be described in detail in the detailed description section which follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for detecting the depth of an asphalt pavement structure according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a calibration plate according to an embodiment of the present application;
FIG. 3 is a graph showing the variation of depth values of a section before and after the tilt error processing according to an embodiment of the present application;
FIG. 4 is a diagram illustrating MAPE statistics corresponding to different texture reference planes when calculating the depth of the texture according to an embodiment of the present application;
Fig. 5 is a schematic diagram illustrating a calculation process of MPD p according to an embodiment of the application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Aiming at the problem of poor efficiency and precision of detecting the construction depth of the asphalt pavement at present, the embodiment of the application provides a method for detecting the construction depth of the asphalt pavement, which comprises the steps of inputting a reference image of the asphalt pavement and n source images shot around the asphalt pavement into a depth map reconstruction model to reconstruct the depth map to obtain the depth map of the reference image, clipping the depth map to obtain an effective area of the depth map by acquiring an absolute depth value of the depth map, fitting the clipped depth map, and correcting an inclination error based on a fitting plane to obtain an accurate depth map of the asphalt pavement so as to obtain the construction depth of the asphalt pavement based on the accurate depth map.
The depth map is obtained by combining a computer vision technology and a depth learning technology, so that the efficiency and the precision of reconstructing the depth map can be greatly enhanced, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be improved, and meanwhile, the efficiency and the precision of detecting the structural depth of the asphalt pavement can be greatly improved because the detection of the structural depth is based on an effective area of the depth map and is completed based on the depth map after inclination error correction.
The method for detecting the depth of the asphalt pavement structure provided by the embodiment of the application is exemplified by the following specific examples.
As shown in fig. 1, the method for detecting the depth of the asphalt pavement structure provided by the embodiment of the application comprises the following steps:
And 11, acquiring images of the asphalt pavement at a plurality of view angles.
The images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement, the images of the multiple visual angles are all RGB images, and RGB represents the colors of three channels of red, green and blue.
The resolution and the number of the images with the plurality of view angles can be properly adjusted according to the performance of the camera. As a preferred example, the resolution of the images of the plurality of view angles may be 3024×3024, the n may be 6, 7, 8, 9, 10, etc., and the camera may be a digital camera such as a camera of a smart phone.
And step 12, inputting images of a plurality of view angles into a depth map reconstruction model to reconstruct the depth map, and obtaining a depth map of the reference image.
The n source images are mainly used for assisting in reconstructing a depth map of the reference image.
In some embodiments of the present application, the depth map reconstruction model may be a conventional depth map reconstruction model. As a preferred example PATCHMATCHNET (PATCHMATCHNET is a deep learning model for image processing) may be used.
The depth map reconstruction model in the above step 12 is a trained model. The training process of the model is described here by way of example PATCHMATCHNET.
Firstly, acquiring image data by adopting a depth camera INTEL REALSENSE D to obtain an RGB-D (RGB-D=common RGB three-channel color image+depth image) training set (covering a common dense graded asphalt concrete mixture (AC), asphalt mastic macadam mixture (SMA), upgrading asphalt mixture pavement such as a water distribution type wearing layer mixture (OGFC) and the like), and carrying out preprocessing operations such as finishing, cutting and the like on the RGB-D training set; the model is then pre-trained using a large-scale public dataset (e.g., DTU dataset), the weight of the pre-trained model is used as an initialization parameter, and then the model is fine-tuned using the pre-processed RGB-D training set.
It is worth mentioning that, by training the model by the above-mentioned training method of transfer learning, the feature and parameter initialization value learned by the pre-training model can be utilized to accelerate the training of the model and improve the performance, and reduce the data amount requirement and training time. The model further learns road texture features from the road texture RGB-D dataset based on the pre-training weights, and the model finally achieves good reconstruction performance.
After model training is finished, ioU indexes (IoU indexes are one standard for measuring accuracy of detecting corresponding objects in a specific dataset) are optionally used for evaluating accuracy of neural network training, and point clouds reconstructed by the model are compared with true point clouds.
The accuracy of the model can be evaluated in particular on an RGB validation set (available for capture with a digital camera). Is defined asV GT∩Pre represents the intersection of the true point cloud and the point cloud reconstructed by the model, V GT∪Pre represents the union of the true point cloud and the point cloud reconstructed by the model, and the true point cloud can be obtained by reconstructing an RGB verification set image through input PhotoScan software.
The IoU index is in a value range of 0 to 1, when the IoU value is closer to 1, the spatial morphology of the 2 three-dimensional models is closer, namely the accuracy of PATCHMATCHNET predicted point cloud is better, and the model can be deduced to reconstruct a depth map with good quality.
In some embodiments of the present application, to improve PATCHMATCHNET the accuracy of reconstructing the depth map, after model training and verification is completed, the impact of different image resolutions and view angle numbers, as well as road surface material types, on the reconstruction quality may also be analyzed.
To ensure that the dataset contains diverse scenes and conditions to obtain a comprehensive analysis result, the analysis can be performed on an RGB validation set containing different image resolutions, number of views, and pavement material types. In a specific analysis procedure, different image resolutions (2048, 3024, 3400) and viewing angle numbers (5, 7, 10) were used for combination and testing. The image resolution (3024) and the number of views (7) are then fixed and tested on different pavement types and the corresponding reconstruction results and evaluation index (IoU index may be used) results recorded.
And comparing the influence of different factors on the reconstruction quality by carrying out statistical analysis on the experimental result. The conclusion is that the reconstruction quality is best under the condition that the image resolution is 3024 multiplied by 3024 and the view angle is 7, the reconstruction quality on different pavement material types is not greatly different, and the effect is stable, so that the reconstruction task in practical application can obtain good effect.
In some embodiments of the application, to demonstrate the accuracy of PATCHMATCHNET in the present application, an RGB validation set was selected as the test subject (specifically screening images with resolution 3024 x 3024, number of perspectives of 7, and different road material types); selecting a traditional three-dimensional reconstruction model colmap as a reference model, using the model to carry out three-dimensional reconstruction on the image in the data set, and obtaining a corresponding reconstruction result; ioU indexes are used for measuring the reconstruction quality of the point cloud under different models.
For the selected RGB validation set, PATCHMATCHNET and conventional three-dimensional reconstruction models are used for reconstruction, respectively. The reconstruction results obtained by the two methods are compared, including visual reconstruction depth map comparison and IoU index comparison in quantification.
The two reconstruction results are quantitatively evaluated using the selected evaluation index. PATCHMATCHNET is better than the conventional three-dimensional reconstruction model in terms of reconstruction quality IoU =0.77 for point clouds, ioU =0.64 for conventional three-dimensional reconstruction models, and far better in terms of reconstruction efficiency than conventional three-dimensional reconstruction models.
And step 13, obtaining an absolute depth value of the depth map, and cutting the depth map to obtain the depth map with the absolute depth value.
In some embodiments of the present application, the absolute depth value of the depth map may be determined using calibration plates of known thickness and cropped to obtain a depth map having the absolute depth value.
And step 14, fitting the depth map with the absolute depth value to obtain a fitting plane.
In some embodiments of the present application, a random consensus sampling (RANSAC) algorithm may be used to fit a depth map with absolute depth values to obtain a fit plane.
And 15, correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement.
In some embodiments of the present application, the accurate depth map of the asphalt pavement may be obtained by obtaining the normal vector n of the fitting plane and then projecting the points in the depth map into the rotated coordinate system through the coordinate transformation matrix T.
And step 16, obtaining the construction depth of the asphalt pavement according to the accurate depth map.
In some embodiments of the application, the depth of construction of the asphalt pavement may be obtained based on a three-dimensional index of volume or a two-dimensional index of profile.
It is worth mentioning that, because the depth map is obtained by combining the computer vision technology and the depth learning technology, the efficiency and the precision of the depth map reconstruction can be greatly enhanced, so that the efficiency and the precision of the detection of the construction depth of the asphalt pavement can be improved, and meanwhile, because the detection of the construction depth is based on the effective area of the depth map and is completed based on the depth map after the inclination error correction, the efficiency and the precision of the detection of the construction depth of the asphalt pavement can be greatly improved.
The specific implementation manner of obtaining the absolute depth value of the depth map and clipping the depth map to obtain the depth map with the absolute depth value is illustrated in the following step 13 with reference to the specific embodiment.
In some embodiments of the present application, four pairs of calibration points may be obtained at four corners inside a calibration plate of known thickness, and then the absolute depth value of the depth map may be obtained by a depth value calculation formula, and the depth map may be cut by using the calibration plate to obtain a depth map having the absolute depth value.
The four corners are in one-to-one correspondence with the four pairs of calibration points, namely, each corner is provided with a pair of calibration points, and the calibration points of the 4 corners are arranged in the same way. To facilitate understanding of the calibration points, the calibration points are illustrated herein by way of example with the calibration plate shown in FIG. 2. Fig. 2 illustrates only a pair of calibration points for one of the four corners, including a circular calibration point on the upper side of the corner and a triangular calibration point on the lower side.
The depth value calculation formula is as follows:
Wherein Z abs represents the absolute depth value of the depth map; z rel represents the relative depth value of the depth map; scale represents the scale factor of the calibration plate, X represents the thickness of the calibration plate, and can be specifically set according to practical values, such as 3 mm; z blue_i represents the relative depth value of one of the ith pair of calibration points (e.g., the circular calibration point in FIG. 2); z red_i represents the relative depth value of another index point (e.g., a triangle index point in FIG. 2) of the ith pair of index points.
It should be noted that, the depth map reconstructed by PATCHMATCHNET can reflect the relative depth values, and Z blue_i and Z red_i refer to the relative depth values in the depth map reconstructed by using PATCHMATCHNET to reconstruct the depth map of the calibration plate.
In some embodiments of the present application, the depth map is cropped using a calibration plate in order to screen out the effective area (also referred to as the range of interest) of the depth map. The size of the depth map after clipping is the same as the size of the calibration plate. As a preferred example, the dimensions of the cropped depth map may be 100mm x 100mm.
The specific implementation of the step 15 to obtain an accurate depth map of the asphalt pavement based on correction of the inclination error of the fitting plane is exemplarily described below with reference to a specific embodiment.
In some embodiments of the present application, the normal vector n of the fitting plane may be obtained first, and then the accurate depth map of the asphalt pavement may be obtained by the inclination correction formula
The inclination correction formula is as follows:
Wherein Z' represents an accurate depth map of the asphalt pavement; t represents a coordinate transformation matrix for describing a matrix form of a transformation relationship from one coordinate system to another coordinate system, which is set according to the selected coordinate system and transformation rule, X and Y represent pixel point coordinates in the depth map; z represents the absolute depth value of the depth map; r represents a rotation matrix,/>Θ represents the angle between the normal vector n and the Z axis, which is the Z axis of the camera (specifically, the camera that can acquire images from multiple angles of view) coordinate system, and is/areT denotes a three-dimensional translation vector, which can be used to describe the translation transformation from one coordinate system to another, which will be translated accordingly on each coordinate axis when adding one vector to the translation vector; /(I)Representing a 3 x1 transpose of the zero vector.
It is worth mentioning that correction of inclination error can effectively correct the inclination error that exists on the road surface, promotes the precision of structure degree of depth. As shown in fig. 3, the absolute difference of the depth values after the inclination error correction becomes smaller, and the overall slope of the cross-sectional depth values becomes smaller, which means that the inclination error of the road surface is effectively corrected. Where the abscissa in fig. 3 represents pixel values of a certain row of the depth map and the ordinate represents depth values relative to the texture reference plane.
The specific implementation of step 16 to obtain the construction depth of the asphalt pavement according to the accurate depth map is exemplarily described below with reference to specific embodiments.
The process for obtaining the construction depth of the asphalt pavement based on the three-dimensional index of the volume comprises the following steps:
By the formula And calculating to obtain the construction depth of the asphalt pavement.
Wherein MTD p represents the construction depth of the asphalt pavement, M and N represent the pixel numbers of the length and width directions of the accurate depth map respectively, Z mn represents the absolute depth value of the nth pixel of the M-th row in the accurate depth map, Z p represents the absolute depth value of the selected texture reference surface,Y represents the area within the calibration plate. It should be noted that, a plane where the p% minimum depth quantile value in the accurate depth map is located may be selected as the texture reference plane.
In some embodiments of the present application, a plane in which a depth quantile value at 5% of an accurate depth map is located may be selected as a texture reference plane, where prediction errors of various types of road surfaces are close to minimum. It will be appreciated that in embodiments of the application, it is not limited what plane of the texture reference surface is specifically the depth map.
To facilitate clarity of the effect of texture references on the texture depth, different texture references are selected to calculate the texture depth, and MAPE statistics are used to measure the average relative error percentage of the texture depth to the measured value.
Wherein, MTD e represents the measured value.
As shown in FIG. 4, tests were performed on 4 different types of asphalt pavements, AC-13, AC-16, SMA-13, OGFC-16, to obtain MAPE statistical indexes corresponding to the construction depths calculated by taking planes at different depth quantile values as texture reference planes. In fig. 4, the abscissa indicates the value of p, and the ordinate indicates the MAPE statistical index value.
The process for obtaining the construction depth of the asphalt pavement based on the two-dimensional index of the section comprises the following steps:
By the formula And calculating to obtain the construction depth of the asphalt pavement.
Wherein MPD p represents the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth,Depth value representing average elevation of jth row of pixels in accurate depth map,/>And the depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
It should be noted that, the depth value of the average elevation and the depth value of the peak elevation may be selected from the calculation process schematic diagram of the MPD p shown in fig. 5. Where the abscissa of fig. 5 represents the pixel values of a certain row of the depth map, the ordinate represents the depth values relative to the texture reference plane, line C represents the peak elevation plane, and line D represents the average elevation plane.
It should be further noted that, in the practical application process, the three-dimensional index based on volume or the two-dimensional index based on section can be selected according to the practical situation to obtain the construction depth of the asphalt pavement.
An exemplary description will be given below of a related apparatus that implements the detection method provided in the present embodiment.
The equipment core components comprise a wide-angle macro lens, a single-lens reflex, a rotary servo motor, a horizontal sliding module, a fixed bracket and the like; the shooting angle of the camera is controlled by the servo motor and the sliding module, and the shooting quantity of the measuring points is 1 (the reference image is shot perpendicular to the road surface) +8 (multi-view image) and is 9 in total. The camera is directly connected with the calculation server, and the photo is exported in real time to conduct pavement structure depth prediction; the light shield is required to be designed, and external light sources such as natural light are replaced by a form of internally provided light sources, so that the influence on imaging results caused by uneven or serious inclination of the ambient light sources is avoided; the equipment is portable and light in operation, and the gravity center deviation of the equipment is avoided.
In summary, the asphalt pavement structure depth detection method provided by the embodiment of the application has the following advantages:
first, non-contact: the non-contact measuring method is a non-contact measuring method, does not need to physically contact the measured object like a sanding method, and is more convenient and safer;
Secondly, enriching information: rich additional information may be provided. In addition to constructing depth values, the depth image may also provide other information about the constructed surface, such as texture, color, etc. Such information may be used for further analysis and applications such as road surface quality assessment, defect detection, etc.
Third, visual display: visual depth images can be generated, and distribution and change conditions of the construction depth values can be intuitively displayed. This facilitates the understanding and analysis of the depth of construction by researchers and decision makers, and supports subsequent decisions and processing.
While the foregoing is directed to the preferred embodiments of the present application, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.
Claims (5)
1. A method for detecting the depth of an asphalt pavement structure, comprising the steps of:
Acquiring images of multiple visual angles of an asphalt pavement; the images of the multiple visual angles comprise a reference image and n source images shot around the asphalt pavement, and the optical axis of a camera shooting the reference image is perpendicular to the asphalt pavement;
Inputting the images of the multiple view angles into a depth map reconstruction model to reconstruct a depth map, and obtaining a depth map of the reference image;
Acquiring an absolute depth value of the depth map, and cutting the depth map to obtain a depth map with the absolute depth value;
fitting the depth map with the absolute depth value to obtain a fitting plane;
Correcting the inclination error based on the fitting plane to obtain an accurate depth map of the asphalt pavement;
acquiring the construction depth of the asphalt pavement according to the accurate depth map;
Wherein the obtaining the absolute depth value of the depth map includes:
Four pairs of calibration points are obtained at four corners inside a calibration plate with known thickness; the four corners are in one-to-one correspondence with the four pairs of calibration points;
Obtaining an absolute depth value of the depth map through a depth value calculation formula;
The depth value calculation formula is as follows:
Wherein Z abs represents the absolute depth value of the depth map, Z rel represents the relative depth value of the depth map, scale represents the scale factor of the calibration plate, x represents the thickness of the calibration plate, Z blue_i represents the relative depth value of one of the ith pair of calibration points, and Z red_i represents the relative depth value of the other of the ith pair of calibration points;
The obtaining the construction depth of the asphalt pavement according to the accurate depth map comprises the following steps:
By the formula Calculating to obtain the construction depth of the asphalt pavement; wherein MTD p represents the construction depth of the asphalt pavement, M and N represent the pixel numbers of the accurate depth map in the length and width directions respectively, Z mn represents the absolute depth value of the nth pixel of the M-th row in the accurate depth map, Z p represents the absolute depth value of the selected texture reference surface,/>Y represents the area in the calibration plate; or alternatively
By the formulaCalculating to obtain the construction depth of the asphalt pavement; wherein MPD p represents the construction depth of the asphalt pavement, N is the total number of pixels of the accurate depth map, MSD represents the average section depth,/>, andDepth value representing average elevation of jth row of pixels in the accurate depth map,/>And a depth value representing the peak elevation of the j-th row of pixels in the accurate depth map.
2. The method of claim 1, wherein cropping the depth map to obtain a depth map having the absolute depth value comprises:
Cutting the depth map by using the calibration plate to obtain a depth map with the absolute depth value; the size of the cut depth map is the same as that of the calibration plate.
3. The method of claim 1, wherein fitting the depth map having the absolute depth values to obtain a fit plane comprises:
and fitting the depth map with the absolute depth value by using a RANSAC algorithm to obtain a fitting plane.
4. The method of claim 1, wherein the correcting for tilt errors based on the fitted plane obtains an accurate depth map of the asphalt pavement, comprising:
obtaining a normal vector n of the fitting plane;
obtaining an accurate depth map of the asphalt pavement through an inclination correction formula;
the inclination correction formula is as follows:
Wherein Z' represents an accurate depth map of the asphalt pavement, T represents a coordinate transformation matrix, X, Y represents pixel coordinates in the depth map, Z represents an absolute depth value of the depth map, R represents a rotation matrix, Θ represents the angle between normal vector n and Z axis,/> T represents a three-dimensional translation vector,/>Representing a 3 x1 transpose of the zero vector.
5. The method of claim 1, wherein the resolution of the images at the plurality of view angles is 3024 x 3024 and n has a value of 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311200907.8A CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311200907.8A CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218089A CN117218089A (en) | 2023-12-12 |
| CN117218089B true CN117218089B (en) | 2024-04-19 |
Family
ID=89043833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311200907.8A Active CN117218089B (en) | 2023-09-18 | 2023-09-18 | Asphalt pavement structure depth detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218089B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112489202A (en) * | 2020-12-08 | 2021-03-12 | 甘肃智通科技工程检测咨询有限公司 | Pavement macroscopic texture reconstruction method based on multi-view deep learning |
| CN112927366A (en) * | 2021-05-10 | 2021-06-08 | 中南大学 | Asphalt pavement structure depth measuring method |
| WO2021115071A1 (en) * | 2019-12-12 | 2021-06-17 | 中国科学院深圳先进技术研究院 | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device |
| CN113066168A (en) * | 2021-04-08 | 2021-07-02 | 云南大学 | Multi-view stereo network three-dimensional reconstruction method and system |
| CN113962858A (en) * | 2021-10-22 | 2022-01-21 | 沈阳工业大学 | Multi-view depth acquisition method |
| CN114280599A (en) * | 2021-11-15 | 2022-04-05 | 江苏金晓电子信息股份有限公司 | Coordinate conversion matching vehicle detection method based on millimeter wave radar and video data |
| CN114812450A (en) * | 2022-04-25 | 2022-07-29 | 山东省路桥集团有限公司 | Machine vision-based asphalt pavement construction uniformity detection and evaluation method |
| CN114994054A (en) * | 2022-05-26 | 2022-09-02 | 哈尔滨工业大学 | Method for determining relation between road surface texture average construction depth and average section depth |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10803546B2 (en) * | 2017-11-03 | 2020-10-13 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
-
2023
- 2023-09-18 CN CN202311200907.8A patent/CN117218089B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021115071A1 (en) * | 2019-12-12 | 2021-06-17 | 中国科学院深圳先进技术研究院 | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device |
| CN112489202A (en) * | 2020-12-08 | 2021-03-12 | 甘肃智通科技工程检测咨询有限公司 | Pavement macroscopic texture reconstruction method based on multi-view deep learning |
| CN113066168A (en) * | 2021-04-08 | 2021-07-02 | 云南大学 | Multi-view stereo network three-dimensional reconstruction method and system |
| CN112927366A (en) * | 2021-05-10 | 2021-06-08 | 中南大学 | Asphalt pavement structure depth measuring method |
| CN113962858A (en) * | 2021-10-22 | 2022-01-21 | 沈阳工业大学 | Multi-view depth acquisition method |
| CN114280599A (en) * | 2021-11-15 | 2022-04-05 | 江苏金晓电子信息股份有限公司 | Coordinate conversion matching vehicle detection method based on millimeter wave radar and video data |
| CN114812450A (en) * | 2022-04-25 | 2022-07-29 | 山东省路桥集团有限公司 | Machine vision-based asphalt pavement construction uniformity detection and evaluation method |
| CN114994054A (en) * | 2022-05-26 | 2022-09-02 | 哈尔滨工业大学 | Method for determining relation between road surface texture average construction depth and average section depth |
Non-Patent Citations (4)
| Title |
|---|
| An improved computation method for asphalt pavement texture depth based on multicular vision 3D reconstruction technology;Han-Cheng Dan 等;《Construction and Building Materials》;20220118;1-12 * |
| Evaluation of asphalt pavement texture using multiview stereo reconstruction based on deep learning;Han-Cheng Dan 等;《Construction and Building Materials》;20240105;1-17 * |
| 基于（图像）三维数据的沥青路面构造深度算法研究;陈鹏;《中国优秀硕士学位论文全文数据库工程科技II辑》;20150315;C034-329 * |
| 沥青路面离析检测算法研究;陈钰闻;《中国优秀硕士学位论文全文数据库工程科技II辑》;20230615;C034-53 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117218089A (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322702B (en) | Intelligent vehicle speed measuring method based on binocular stereo vision system | |
| CN112927366B (en) | Asphalt pavement structure depth measuring method | |
| Vaudrey et al. | Differences between stereo and motion behaviour on synthetic and real-world stereo sequences | |
| CN101354241B (en) | Method for evaluating aggregate digital image | |
| CN106978774B (en) | A kind of road surface pit slot automatic testing method | |
| CN104574393A (en) | Three-dimensional pavement crack image generation system and method | |
| CN108107462A (en) | The traffic sign bar gesture monitoring device and method that RTK is combined with high speed camera | |
| CN111192235A (en) | Image measuring method based on monocular vision model and perspective transformation | |
| CN105261018A (en) | Visibility detection method based on optical model and dark primary color priori theory | |
| CN107843204A (en) | Side slope three-dimensional deformation monitoring method and system based on monitoring level video camera | |
| CN110782498B (en) | Rapid universal calibration method for visual sensing network | |
| Dan et al. | An improved computation method for asphalt pavement texture depth based on multiocular vision 3D reconstruction technology | |
| CN111473776A (en) | Landslide crack monitoring method based on single-image close-range photogrammetry | |
| CN111179335A (en) | Standing tree measuring method based on binocular vision | |
| CN116342674A (en) | Method for calculating asphalt pavement construction depth by three-dimensional model | |
| CN114972177A (en) | Road disease identification management method and device and intelligent terminal | |
| CN110702343B (en) | Deflection measurement system and method based on stereoscopic vision | |
| CN117218089B (en) | Asphalt pavement structure depth detection method | |
| Guan et al. | Multi-scale asphalt pavement deformation detection and measurement based on machine learning of full field-of-view digital surface data | |
| Li et al. | Road Damage Evaluation via Stereo Camera and Deep Learning Neural Network | |
| CN105046703A (en) | Image analysis atmospheric visibility method | |
| CN114283121A (en) | Paving width detection method and device, readable storage medium and paver | |
| CN111429430B (en) | Tunnel detection vehicle size mapping relation calculation method based on machine vision | |
| Chou et al. | Road marking retroreflectivity study via a visual algorithm | |
| CN112907739B (en) | Method, device and system for acquiring height difference information of well lid |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |