CN115861408A - Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof - Google Patents
Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof Download PDFInfo
- Publication number
- CN115861408A CN115861408A CN202211581470.2A CN202211581470A CN115861408A CN 115861408 A CN115861408 A CN 115861408A CN 202211581470 A CN202211581470 A CN 202211581470A CN 115861408 A CN115861408 A CN 115861408A
- Authority
- CN
- China
- Prior art keywords
- pit
- unmanned aerial
- aerial vehicle
- target
- laser
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application provides a method for unmanned aerial vehicle to inspect road surface pot holes based on laser point tracking and an application thereof, and the method comprises the following steps: s00, according to the real-time height of the unmanned aerial vehicle, dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture to obtain calibration data; s10, shooting and detecting the road surface through an unmanned aerial vehicle, identifying the pit diseases and marking the pit diseases as target pits; s20, tracking the target pit, flying to the upper side of the target pit, and shooting to obtain a pit picture; s30, calculating the actual area of the target pit; s40, tracking and identifying the position of the laser point, and calculating the overlapping degree of the actual area of the laser point and the actual area of the pit hole according to the position of the laser point so as to set the flight track of the unmanned aerial vehicle; s50, calculating the depth of the target pot hole; and S60, calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole. The application has the advantages of low detection cost and good detection effect.
Description
Technical Field
The application relates to the technical field of pavement pot hole identification, in particular to a method for inspecting pavement pot holes by an unmanned aerial vehicle based on laser point tracking and application thereof.
Background
In order to ensure the safe driving of vehicles on roads and prolong the service life of roads, the roads need to be detected and maintained regularly, and the detection and identification of three-dimensional diseases of the road surfaces is a necessary and fundamental work.
At present, the advanced means is that an unmanned aerial vehicle carries on an acquisition device to carry out inspection and identification, flies and acquires road surface images to analyze, and geometric information and position information of the pit are obtained. However, when analyzing the acquired road surface image data, only the image processing module is described to process the road section image to obtain the geometric parameters and the position parameters of the pothole, but no detailed scheme is provided to describe the calculation of the area and the depth of the pothole, so that the reference value is limited. And the depth information calculation mode is to judge the depth of the pit hole through point cloud or laser radar, and the equipment cost is high, so that the detection cost is high. The other part is to judge whether the pit is a hole or not only by an apparent image, and lacks depth information.
In summary, a method for unmanned aerial vehicle to inspect pits on a road surface based on laser point tracking and an application thereof, which meet requirements and reduce detection cost, are urgently needed.
Disclosure of Invention
The embodiment of the application provides a method for inspecting pits in a road surface by an unmanned aerial vehicle based on laser point tracking and application thereof, and aims to solve the problems of high detection cost or lack of depth information in the prior art.
The core technology of the invention is mainly based on image recognition and laser point tracking to realize pit hole recognition.
In a first aspect, the application provides a method for unmanned aerial vehicle to inspect pits on a road surface based on laser point tracking, and the method comprises the following steps:
s00, according to the real-time height of the unmanned aerial vehicle, dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture to obtain calibration data;
s10, shooting and detecting the road surface through an unmanned aerial vehicle, identifying the pit diseases and marking the pit diseases as target pits;
s20, tracking a target pit, flying above the target pit for shooting, and obtaining a pit picture;
s30, analyzing the pit picture, marking the pit characteristics, and calculating the actual area of the target pit by combining calibration data according to the number of pixel points occupied by the pit characteristics;
s40, generating a pit recognition frame on the periphery of the pit through a pit target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pit recognition frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle;
when the unmanned aerial vehicle shoots the target pot hole, the unmanned aerial vehicle simultaneously aims at the target pot hole to carry out laser identification and distance measurement operation;
s50, according to the flight path, sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is larger than a set threshold value, solving a first average value, simultaneously extracting the laser ranging lengths of all acquisition points of which the overlapping degree is smaller than or equal to the set threshold value, solving a second average value, and calculating the depth of the target pit according to the first average value and the second average value;
and S60, calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
Further, in the step S20, the shooting height of the target pothole that flies above the target pothole is one of the calibration data.
Further, in the step S30, the pothole picture is automatically analyzed through the PSPNet semantic segmentation algorithm, the pothole feature is marked, the number of pixel points occupied by the pothole feature is analyzed, and the actual area of the target pothole is calculated according to the actual area of the photographed road surface of each pixel point in the road surface picture corresponding to the current hovering height of the unmanned aerial vehicle.
Further, in step S30, a pit recognition frame of a circumscribed rectangle is generated according to the actual area of the target pit, and the pit recognition frame includes coordinates of four vertices and a center point.
Further, in the step S40, longitude and latitude coordinates of four vertexes and a center point of the pit recognition frame are obtained, the unmanned aerial vehicle sequentially flies above all the points to perform hovering shooting, and laser ranging lengths are recorded respectively.
Further, in the step S50, in the flight process, by shooting the real-time road surface image, and simultaneously recognizing with the pit semantic segmentation algorithm and the laser point semantic segmentation algorithm, respectively generating a pit segmentation recognition map and a laser point segmentation recognition map, and binarizing the pit segmentation recognition map and the laser point segmentation recognition map, performing pixel point coincidence calculation to obtain an intersection ratio of the actual area of the pit and the actual area of the laser point, where the intersection ratio is the overlapping degree.
Further, in the step S50, it is determined that, in 1 frame of picture, when the intersection ratio of the actual area of the pit and the actual area of the laser point is not less than 60%, the distance of the laser point acquired by the unmanned aerial vehicle at this time is the distance acquired by just flying within the actual area of the pit.
In a second aspect, the application provides a device based on unmanned aerial vehicle patrols and examines road surface pot hole includes:
the control module is used for controlling the unmanned aerial vehicle; the unmanned aerial vehicle is used for controlling the unmanned aerial vehicle to fly above the target pothole for shooting after tracking and identifying the target pothole, and a pothole picture is obtained;
an unmanned aerial vehicle as a vehicle;
the high-definition camera is used for shooting road surface images;
the edge calculation box is used for identifying the pit diseases in the shot pavement images and marking the pit diseases as target pits; tracking the target pit; analyzing the pit picture, marking pit characteristics, and calculating the actual area of the target pit by combining calibration data according to the number of pixel points occupied by the pit characteristics; generating a pit recognition frame on the periphery of the pit through a pit target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pit recognition frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle; according to the flight path, sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is greater than a set threshold value, solving a first average value, and simultaneously extracting the laser ranging of all acquisition points of which the overlapping degree is less than or equal to the set threshold value; wherein the flight path is generated on the basis of the vertexes of the four corners of the pit recognition frame;
the laser range finder is used for measuring the depth of the pit; when the unmanned aerial vehicle shoots the target pothole, the unmanned aerial vehicle simultaneously aims at the target pothole to carry out laser identification and distance measurement operation;
the calculation output module is used for dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture according to the real-time height of the unmanned aerial vehicle so as to obtain calibration data; the system comprises a laser distance measuring device, a first average value, a second average value and a third average value, wherein the laser distance measuring device is used for sequentially extracting the laser distance measuring lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of a laser point during laser identification distance measuring is larger than a set threshold value according to a flight path, simultaneously extracting the laser distance measuring lengths of all acquisition points of which the overlapping degree is smaller than or equal to the set threshold value, calculating the second average value and calculating the depth of a target pit according to the first average value and the second average value; and calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
In a third aspect, the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor is configured to run the computer program to execute the above method for inspecting a pothole on a road surface by an unmanned aerial vehicle based on laser point tracking.
In a fourth aspect, the present application provides a readable storage medium having stored thereon a computer program comprising program code for controlling a process to execute a process, the process comprising a method for inspecting a roadway pothole by a laser spot tracking based drone according to the above.
The main contributions and innovation points of the invention are as follows: 1. compared with the prior art, the method and the device have the advantages that the depth of the pot hole is not required to be judged through point cloud or laser radar, so that the equipment cost is relatively low, the detection cost is also relatively low, meanwhile, whether the pot hole is judged by only depending on image identification, the image identification and the laser ranging are combined, and the depth information of the pot hole is obtained, so that the basic condition that the pot hole is judged by combining the apparent characteristic with the depth information is met, in addition, the maintenance idea is realized according to the main diseases of tight grip of the urban road, and the detection cost is greatly reduced;
2. compared with the prior art, the method and the system have the advantages that in the process of daily urban road inspection by the unmanned aerial vehicle, the characteristics of the pot holes can be quickly locked, the apparent information and the depth information of the pot holes are timely acquired through a semantic segmentation algorithm and a target tracking algorithm, and then whether the pot holes are serious or not is judged, and data support is provided for subsequent maintenance;
3. compared with the existing binocular recognition and detection method of the pot holes, the binocular scheme refers to the depth of the lens and the target object, but not the depth of the pot holes. And the depth of the pit is obtained according to the difference value of the depths of the two lenses. Therefore, the cost is higher, the binocular camera can better restore the site details under different optical conditions by depending on a good processor, the performance requirement on the signal processor is higher, and the cost is invisibly increased. And the performance requirement of the signal processor is lower and the cost is lower.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flow of a method for inspecting a pit hole on a road surface by an unmanned aerial vehicle based on laser spot tracking according to an embodiment of the application;
FIG. 2 is a schematic view of a preferred flight path according to an embodiment of the present application
Fig. 3 is a schematic view of a pothole identification box and another flight path in accordance with an embodiment of the present application;
FIG. 4 is a schematic view of a laser identification box of an embodiment of the present application;
fig. 5 is a pit and laser spot segmentation recognition chart pic1 according to an embodiment of the present application;
fig. 6 is a pit hole binarization image pic2 in the embodiment of the present application;
fig. 7 is a laser dot binarization image pic2 of the embodiment of the present application;
fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of one or more embodiments of the specification, as detailed in the claims that follow.
It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described herein. In some other embodiments, the method may include more or fewer steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.
And (3) judging the pits according to the local maintenance standard, wherein the area of each pit is more than 0.04 square meter, and the depth of each pit is more than 10 mm. The current mode of judging whether the pit is a hole is partly to judge whether the pit is a hole through the apparent characteristics and the depth information. And the depth information calculation mode is to judge the depth of the pit hole through point cloud or laser radar, and the equipment cost is high, so that the detection cost is high. The other part is to judge whether the pit is a hole or not only by an apparent image, and lacks depth information.
Based on this, the present invention solves the above problems based on a combination of image recognition and laser ranging.
Example one
The application aims at providing an unmanned aerial vehicle patrols and examines road surface pothole method based on laser point tracking, mainly carry on high definition camera (shoot road surface pothole) for unmanned aerial vehicle, edge calculation box (the target detection algorithm of deployment pothole in the edge calculation box, target tracking algorithm, pothole segmentation recognition algorithm), laser range finder (install perpendicularly and be used for measuring the pothole degree of depth), and fly according to fixed airline, shoot the road surface and the automatic pothole disease that discovers of algorithm, can solve the problem that prior art exists, specifically, refer to figure 1, the method specifically includes following steps:
s00, dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture according to the real-time height of the unmanned aerial vehicle to obtain calibration data;
in this embodiment, the calibration is performed by calibrating that the unmanned aerial vehicle hovers at different heights to shoot the road surface, that is, the actual area of the road surface shot by each pixel point in each picture of the shot picture is dynamically calculated according to the real-time vertical distance between the unmanned aerial vehicle and the ground.
S10, shooting and detecting the road surface through an unmanned aerial vehicle, identifying pit hole diseases and marking the pit hole diseases as a target pit hole;
s20, tracking the target pit, flying to the upper side of the target pit, and shooting to obtain a pit picture;
in this embodiment, when the unmanned aerial vehicle is cruising in the air, when a pothole is found on the road surface by using a target detection algorithm of the pothole, the unmanned aerial vehicle flies above the pothole through a pothole tracking algorithm, selects a proper hovering height (the height is preferably one of the heights during the previous calibration, so that the actual area of the photographed road surface of each pixel point can be quickly obtained), and shoots image data of the target pothole.
S30, analyzing the pothole picture, marking pothole characteristics, and calculating the actual area of the target pothole by combining calibration data according to the number of pixel points occupied by the pothole characteristics;
in this embodiment, a PSPNet semantic segmentation algorithm deployed in an edge box is automatically started to perform automatic analysis of a pothole picture, a pothole feature is marked, the number of pixel points occupied by the pothole feature is analyzed, and the actual area of the pothole is calculated according to the actual area of the road surface photographed by each pixel point in the road surface picture corresponding to the current hovering height of the unmanned aerial vehicle.
S40, generating a pothole identification frame on the periphery of the pothole through a pothole target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pothole identification frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle;
when the unmanned aerial vehicle shoots the target pot hole, the unmanned aerial vehicle simultaneously aims at the target pot hole to carry out laser identification and distance measurement operation;
data acquisition: as shown in fig. 2 and 4, when the unmanned aerial vehicle flies above the pothole through the pothole tracking algorithm, according to the boundary box information of the pothole identification box:
# Upper left corner xmin
# Upper left corner ymin
Bottom right corner xmax
Bottom right corner ymax
Preferably, a deep sort multi-target tracking algorithm is adopted to continuously track and capture the characteristics of the pot holes and the laser points thereof (cross-correlation method), an id is respectively assigned to the characteristics of the pot holes and the laser points of each frame in the video, and the behavior track of each id is tracked in real time. According to the positions of all vertexes of the boundary frame of the hole identification frame in the picture, and the adjustment of the flight direction is judged, and when the IOU (all called Intersection over Unit, in this embodiment, overlapping degree) of the actual area region of the hole and the actual area region of the laser point is more than or equal to 60% in 1 frame of picture, the laser ranging value corresponding to the picture shooting time is defaulted to be the distance between the current laser point and the position of the hole.
Preferably, a pothole identification box is generated at the periphery of the pothole according to a pothole target detection algorithm, and the longitude and latitude coordinate position of each vertex is obtained according to each vertex of the boundary box. And generating a collection route through route planning.
As shown in fig. 2, the flight path: and (4) taking the default point A as a collection starting point, reaching the actual position of the point A according to the longitude and latitude coordinates of the point A, carrying out hovering shooting, and carrying out hovering shooting when the point A moves 0.1m in the direction from A to B and along the direction indicated by the step. When the point B is reached, the flying direction is adjusted, the direction from B to D is taken as the second flying direction, and the like, from S1 to S17, and finally to the point D.
Specifically, a real-time road surface image is shot, and as shown in fig. 5-7, the real-time road surface image is identified by a pit hole + laser point semantic segmentation algorithm, so that a pit hole segmentation identification map pic1 and a laser point segmentation identification map pic1 are respectively generated. And processing the pit hole + laser point segmentation map pic1 into binary maps, namely a pit hole binary map pic2 and a laser point binary map pic2. The pixel points at the pit binary map pic2 pit characteristic positions are all 1, and the pixel points at the other positions are all 0; in the laser point binary image pic3, the pixel points at the characteristic positions of the laser points are all 1, and the pixel points at the other positions are all 0. And carrying out pixel point coincidence calculation on the pit hole binary image pic2 and the laser point binary image pic3 to obtain the intersection ratio of the actual area of the pit hole and the actual area of the laser point. And dividing the shot image containing the pit and the laser irradiation point in real time to generate an intersection ratio IOU between the actual area of the pit and the actual area of the laser point. In 1 frame of picture, when the Intersection ratio IOU (all called Intersection over Union, in this embodiment, may be called overlap degree) of the actual area of the hole and the actual area of the laser spot is not less than 60%, the distance of the laser spot acquired by the unmanned aerial vehicle at this time is the distance acquired by just flying in the actual area of the hole by default.
Preferably, as shown in fig. 3, (a rectangular frame center point method, which is the most basic method and does not need to calculate the overlapping degree), according to the calibration result in the step S00, the real longitude and latitude coordinates of the top points of the four points a, B, C, D and the point F of the 1 middle point are calculated and obtained, so that the unmanned aerial vehicle flies according to a → B → C → D → F and hovers and shoots when reaching the five points a, B, C, D and F, and the distance length of the laser range finder is recorded.
S50, sequentially extracting the laser ranging lengths of all the acquisition points with the overlapping degree larger than a set threshold value according to the flight path, solving a first average value, simultaneously extracting the laser ranging lengths of all the acquisition points with the overlapping degree smaller than or equal to the set threshold value, solving a second average value, and calculating the depth of the target pit hole according to the first average value and the second average value;
in this embodiment, in the data acquisition process (cross-correlation method), the edge of the pit and the position of the laser are tracked and determined in real time, when the IOU between the actual area of the edge of the pit and the actual area of the laser point is greater than or equal to 60%, that is, the laser point is in the pit recognition frame, the distance obtained by the distance measurement of the laser point at the position is defaulted to be a1 as a basis for further measuring the depth of the pit, and so on. The next data acquisition point is the position which is in the same direction as the previous flight direction and is separated by 0.1 m; when the actual area of the edge of the pit and the actual area of the laser point are less than 60 percent, namely the distance between the laser point acquired by the unmanned aerial vehicle is the distance acquired just outside the actual area of the pit in flying, the distance obtained by measuring the distance of the laser point at the position is defaulted to be used as the height from the laser range finder to the normal road surface to be further measured and is recorded as b1, and the like. The next data acquisition point is the position opposite to the previous flight direction and spaced by 0.1 m; and by analogy, the laser ranging values of the effective acquisition points in the range of the whole pit tracking identification frame are finally acquired.
The length of the laser range finder for extracting all acquisition points with IOU more than or equal to 60% is calculated by all A1, a2, a3, a4 \8230; \ 8230;. Average A1 is obtained. The length B1, B2, B3, B4 of the laser range finder for extracting all the collection points with the IOU less than 60 percent, 8230, and the average B1 is obtained. The depth of the target pit can be obtained through A1-B1.
Preferably, as shown in fig. 3, (rectangular frame center point method) an average A1 is obtained as the distance from the laser emitting point to the normal road surface from all A1, a2, a3, a4 \8230; (8230); (rectangular frame center point method) according to the lengths of the a, B, C, D four-point laser range finders. The length of the point F laser distance meter in the target frame is B1, which is the distance between the pit and the laser emission point. The depth of the pit can be obtained from A1-B1. The unmanned aerial vehicle flies to four points A, B, C and D respectively, laser ranging distances are measured, namely the distances from the laser transmitters to the normal road surface are the distances A1, a2, a3 and a4, and the average number A1 is obtained. And the unmanned aerial vehicle flies to the point E respectively to measure the laser ranging distance, namely the distance from the laser transmitter to the pit hole, namely B1. The depth of the pit can be obtained from A1-B1.
And S60, calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
In this embodiment, it is determined that the maintenance requirement is met according to the area and the depth of the pot hole, that is, the pot hole is calculated when the road surface is damaged to form the pot hole with a depth of more than 10mm and the area is more than 0.04 square meters. And then, calculating the breakage rate DR of each kilometer evaluation section by combining other diseases of the road surface (specifically how to calculate the damage rate DR as the prior art, the application only provides data), wherein the PCI (vector condition index) represents an index of the integrity degree of the road surface, the larger the value is, the better the road condition is, and the more specifically how to calculate the damage rate DR as the prior art, the application only provides data). And finally outputting a road surface technical condition evaluation report.
Example two
Based on the same design, this application has still provided a device based on unmanned aerial vehicle patrols and examines road surface pot hole, includes:
the control module is used for controlling the unmanned aerial vehicle; the unmanned aerial vehicle is used for controlling the unmanned aerial vehicle to fly above the target pothole for shooting after tracking and identifying the target pothole, and a pothole picture is obtained;
an unmanned aerial vehicle as a vehicle;
the high-definition camera is used for shooting road surface images;
the edge calculation box is used for identifying the pit diseases in the shot pavement images and marking the pit diseases as target pits; tracking the target pit; analyzing the pit picture, marking pit characteristics, and calculating the actual area of the target pit by combining calibration data according to the number of pixel points occupied by the pit characteristics; generating a pit recognition frame at the periphery of a pit through a pit target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pit recognition frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle;
the laser range finder is used for measuring the depth of the pit;
the calculation output module is used for dynamically calculating the actual area of the photographed road surface of each pixel point in each photographed picture according to the real-time height of the unmanned aerial vehicle so as to obtain calibration data; according to the flight path, sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is greater than a set threshold value, calculating a first average value, simultaneously extracting the laser ranging lengths of all acquisition points of which the overlapping degree is less than or equal to the set threshold value, calculating a second average value, and calculating the depth of the target pit according to the first average value and the second average value; and calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
EXAMPLE III
The present embodiment also provides an electronic device, referring to fig. 8, comprising a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to execute the computer program to perform the steps in any of the above method embodiments.
Specifically, the processor 402 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the above-described methods for inspecting a pothole on a road surface by a laser point tracking-based drone.
Optionally, the electronic apparatus may further include a transmission device 406 and an input/output device 408, where the transmission device 406 is connected to the processor 402, and the input/output device 408 is connected to the processor 402.
The transmitting device 406 may be used to receive or transmit data via a network. Specific examples of the network described above may include wired or wireless networks provided by communication providers of the electronic devices. In one example, the transmission device includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmitting device 406 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
The input and output devices 408 are used to input or output information. In the present embodiment, the input information may be a detection instruction or the like, and the output information may be a road surface damage assessment report or the like.
Example four
The embodiment also provides a readable storage medium, wherein a computer program is stored in the readable storage medium, the computer program comprises program codes for controlling a process to execute the process, and the process comprises the method for the unmanned aerial vehicle to inspect the pits on the road surface based on the laser point tracking according to the first embodiment.
It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.
In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
Embodiments of the invention may be implemented by computer software executable by a data processor of the mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and/or macros can be stored in any device-readable data storage medium and they include program instructions for performing particular tasks. The computer program product may comprise one or more computer-executable components configured to perform embodiments when the program is run. The one or more computer-executable components may be at least one software code or a portion thereof. Further in this regard it should be noted that any block of the logic flow as in the figures may represent a program step, or an interconnected logic circuit, block and function, or a combination of a program step and a logic circuit, block and function. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. The physical medium is a non-transitory medium.
It should be understood by those skilled in the art that various features of the above embodiments can be combined arbitrarily, and for the sake of brevity, all possible combinations of the features in the above embodiments are not described, but should be considered as within the scope of the present disclosure as long as there is no contradiction between the combinations of the features.
The above examples are merely illustrative of several embodiments of the present application, and the description is more specific and detailed, but not to be construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. The method for inspecting the pit holes on the road surface by the unmanned aerial vehicle based on laser point tracking is characterized by comprising the following steps:
s00, according to the real-time height of the unmanned aerial vehicle, dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture to obtain calibration data;
s10, shooting and detecting the road surface through an unmanned aerial vehicle, identifying pit hole diseases and marking the pit hole diseases as a target pit hole;
s20, tracking the target pot hole, flying to the position above the target pot hole for shooting, and obtaining a pot hole picture;
s30, analyzing the pit picture, marking pit characteristics, and calculating the actual area of the target pit by combining the calibration data according to the number of pixel points occupied by the pit characteristics;
s40, generating a pit recognition frame on the periphery of a pit through a pit target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pit recognition frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle;
when the unmanned aerial vehicle shoots the target pot hole, the unmanned aerial vehicle simultaneously aims at the target pot hole to carry out laser identification and distance measurement operation;
s50, according to the flight path, sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is larger than a set threshold value, calculating a first average value, extracting the laser ranging lengths of all acquisition points of which the overlapping degree is smaller than or equal to the set threshold value, calculating a second average value, and calculating the depth of the target pit according to the first average value and the second average value;
and S60, calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
2. The unmanned aerial vehicle inspection road surface pot hole method based on laser point tracking according to claim 1, wherein in the step S20, a shooting height of shooting while flying above the target pot hole is one of the calibration data.
3. The unmanned aerial vehicle inspection road surface pot hole method based on laser point tracking according to claim 1, wherein in step S30, the pot hole picture is automatically analyzed through a PSPNet semantic segmentation algorithm, the pot hole characteristics are marked, the number of pixel points occupied by the pot hole characteristics is analyzed, and the actual area of the target pot hole is calculated according to the actual area of the photographed road surface of each pixel point in the road surface picture corresponding to the current hovering height of the unmanned aerial vehicle.
4. The unmanned aerial vehicle inspection tour road pit method based on laser spot tracking of claim 1, wherein in step S30, a pit recognition frame of a circumscribed rectangle is generated according to the actual area of the target pit, and the pit recognition frame contains coordinates of four vertexes and a center point.
5. The unmanned aerial vehicle inspection tour road surface pot hole method based on laser spot tracking of claim 4, wherein in step S40, longitude and latitude coordinates of four vertexes and a center point of the pot hole identification frame are obtained, and the unmanned aerial vehicle flies above all points in sequence to perform hovering shooting, and laser ranging lengths are recorded respectively.
6. The unmanned aerial vehicle inspection road surface pothole method based on laser point tracking according to claim 1, wherein in the step S50, a pothole division recognition graph and a laser point division recognition graph are respectively generated by shooting a real-time road surface image and simultaneously recognizing by a pothole semantic division algorithm and a laser point semantic division algorithm in a flying process, binarization processing is performed on the pothole division recognition graph and the laser point division recognition graph, pixel point coincidence calculation is performed to obtain a cross-over ratio of a pothole actual area and a laser point actual area, and the cross-over ratio is an overlapping degree.
7. The method for unmanned aerial vehicle inspection of roadway potholes based on laser point tracking according to claim 6, wherein in the step S50, when the intersection ratio of the actual area of the pothole and the actual area of the laser point is more than or equal to 60% in 1 frame of picture, the distance of the laser point acquired by the unmanned aerial vehicle at the moment is the distance acquired by just flying in the actual area of the pothole by default.
8. The utility model provides a device based on unmanned aerial vehicle patrols and examines road surface pot hole which characterized in that includes:
the control module is used for controlling the unmanned aerial vehicle; the unmanned aerial vehicle is used for controlling the unmanned aerial vehicle to fly above the target pothole for shooting after tracking and identifying the target pothole, and a pothole picture is obtained;
an unmanned aerial vehicle as a vehicle;
the high-definition camera is used for shooting road surface images;
the edge calculation box is used for identifying the pit diseases in the shot pavement image and marking the pit diseases as target pits; tracking the target pit; analyzing the pit picture, marking pit characteristics, and calculating the actual area of the target pit by combining calibration data according to the number of pixel points occupied by the pit characteristics; generating a pit recognition frame at the periphery of a pit through a pit target detection algorithm, acquiring the longitude and latitude coordinate position of each vertex according to each vertex of the pit recognition frame, and generating an acquisition route through route planning to set the flight track of the unmanned aerial vehicle; according to the flight path, sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is greater than a set threshold value, calculating a first average value, simultaneously extracting the laser ranging lengths of all acquisition points of which the overlapping degree is less than or equal to the set threshold value, calculating a second average value, and calculating the depth of the target pit according to the first average value and the second average value;
wherein the flight path is generated on the basis of the vertexes of the four corners of the pit recognition frame;
the laser range finder is used for measuring the depth of the pit; when the unmanned aerial vehicle shoots the target pothole, the unmanned aerial vehicle simultaneously aims at the target pothole to carry out laser identification and distance measurement operation;
the calculation output module is used for dynamically calculating the actual area of the photographed road surface of each pixel point in each picture of the photographed picture according to the real-time height of the unmanned aerial vehicle so as to obtain calibration data; the system is used for sequentially extracting the laser ranging lengths of all acquisition points of which the overlapping degree of the actual area of the pit and the actual area of the laser point during laser identification ranging is larger than a set threshold value according to the flight path, solving a first average value, simultaneously extracting the laser ranging lengths of all acquisition points of which the overlapping degree is smaller than or equal to the set threshold value, solving a second average value, and calculating the depth of a target pit according to the first average value and the second average value; and calculating and outputting the technical condition data of the road surface based on the depth and the actual area of the target pot hole.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the method for unmanned aerial vehicle inspection of road pits based on laser point tracking according to any one of claims 1 to 7.
10. A readable storage medium having stored therein a computer program comprising program code for controlling a process to execute a process, the process comprising the method of laser point tracking based unmanned aerial vehicle inspection of road pits according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211581470.2A CN115861408A (en) | 2022-12-09 | 2022-12-09 | Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211581470.2A CN115861408A (en) | 2022-12-09 | 2022-12-09 | Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115861408A true CN115861408A (en) | 2023-03-28 |
Family
ID=85671617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211581470.2A Pending CN115861408A (en) | 2022-12-09 | 2022-12-09 | Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861408A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116433701A (en) * | 2023-06-15 | 2023-07-14 | 武汉中观自动化科技有限公司 | Workpiece hole profile extraction method, device, equipment and storage medium |
| CN117420143A (en) * | 2023-12-19 | 2024-01-19 | 斯润天朗（北京）科技有限公司 | Road surface defect detection method and device based on multi-source sensor fusion and computer equipment |
-
2022
- 2022-12-09 CN CN202211581470.2A patent/CN115861408A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116433701A (en) * | 2023-06-15 | 2023-07-14 | 武汉中观自动化科技有限公司 | Workpiece hole profile extraction method, device, equipment and storage medium |
| CN116433701B (en) * | 2023-06-15 | 2023-10-10 | 武汉中观自动化科技有限公司 | Workpiece hole profile extraction method, device, equipment and storage medium |
| CN117420143A (en) * | 2023-12-19 | 2024-01-19 | 斯润天朗（北京）科技有限公司 | Road surface defect detection method and device based on multi-source sensor fusion and computer equipment |
| CN117420143B (en) * | 2023-12-19 | 2024-03-15 | 斯润天朗（北京）科技有限公司 | Road surface defect detection method and device based on multi-source sensor fusion and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10846874B2 (en) | Method and apparatus for processing point cloud data and storage medium | |
| US10198632B2 (en) | Survey data processing device, survey data processing method, and survey data processing program | |
| CN115861408A (en) | Unmanned aerial vehicle road surface pit inspection method based on laser point tracking and application thereof | |
| US20210342620A1 (en) | Geographic object detection apparatus and geographic object detection method | |
| Puente et al. | Automatic detection of road tunnel luminaires using a mobile LiDAR system | |
| CN111753757B (en) | Image recognition processing method and device | |
| CN115331190B (en) | Road hidden danger identification method and device based on radar vision fusion | |
| CN113724259B (en) | Well lid abnormity detection method and device and application thereof | |
| CN114445803A (en) | Driving data processing method and device and electronic equipment | |
| CN111897906A (en) | Method, device, equipment and storage medium for processing map data | |
| US20200279395A1 (en) | Method and system for enhanced sensing capabilities for vehicles | |
| CN114252884A (en) | Method and device for positioning and monitoring roadside radar, computer equipment and storage medium | |
| CN113505769B (en) | Target detection method and vehicle throwing and dripping identification method applying same | |
| CN114722944A (en) | Point cloud precision determination method, electronic device and computer storage medium | |
| Kukko et al. | Mobile mapping system and computing methods for modelling of road environment | |
| JP2020160840A (en) | Road surface defect detecting apparatus, road surface defect detecting method, road surface defect detecting program | |
| CN114252859A (en) | Target area determination method and device, computer equipment and storage medium | |
| US20230314169A1 (en) | Method and apparatus for generating map data, and non-transitory computer-readable storage medium | |
| CN116430404A (en) | Method and device for determining relative position, storage medium and electronic device | |
| CN115797310A (en) | Method for determining inclination angle of photovoltaic power station group string and electronic equipment | |
| CN115830481A (en) | Method for inspecting road surface pot holes based on unmanned aerial vehicle and application thereof | |
| CN114782496A (en) | Object tracking method and device, storage medium and electronic device | |
| WO2021138372A1 (en) | Feature coverage analysis | |
| CN114252868A (en) | Laser radar calibration method and device, computer equipment and storage medium | |
| CN115546704B (en) | Vehicle projectile identification method, device and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |