CN109902668B - Unmanned aerial vehicle-mounted road surface detection system and detection method - Google Patents
Unmanned aerial vehicle-mounted road surface detection system and detection method Download PDFInfo
- Publication number
- CN109902668B CN109902668B CN201910305803.0A CN201910305803A CN109902668B CN 109902668 B CN109902668 B CN 109902668B CN 201910305803 A CN201910305803 A CN 201910305803A CN 109902668 B CN109902668 B CN 109902668B
- Authority
- CN
- China
- Prior art keywords
- image
- unmanned aerial
- aerial vehicle
- gray
- road surface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an unmanned aerial vehicle-mounted road surface detection system and a detection method, wherein the unmanned aerial vehicle-mounted road surface detection system comprises an unmanned aerial vehicle, and a GPS system and an unmanned aerial vehicle flight height control system are arranged on the unmanned aerial vehicle; the unmanned aerial vehicle is provided with a camera, and the camera is connected with an image sensor; the output end of the image sensor is connected to the input end of the signal amplification circuit, and the output end of the signal amplification circuit is connected to the input end of the signal conversion circuit; the output end of the signal conversion circuit is connected to a wireless data transmission station; the input end of the signal amplification circuit is connected with the output end of the signal detection and processing circuit, and the input end of the signal detection and processing circuit is connected with the controller; the ground wireless data transmission radio station is connected with the ground server through the level adapter plate, and the wireless data transmission radio station carried on the unmanned aerial vehicle is communicated with the ground wireless data transmission radio station in a wireless mode. Different road surface information can be collected and image processing can be respectively carried out, corresponding models are established, and the road surface can be dynamically detected in real time.
Description
Technical Field
The disclosure relates to the technical field of traffic road surface detection, in particular to an unmanned aerial vehicle-mounted road surface detection system and a detection method.
Background
With the development of road traffic, road traffic plays an important role in economic development and brings great convenience to people going out. However, the accompanying problems are also particularly prominent, such as sinking of the road surface, breakage of the road surface, scattering of objects, untimely cleaning of solid wastes generated in construction, small animals staying in the middle of the road and the like, which seriously affect the normal running of vehicles and cause a great number of traffic accidents. At present, a part of domestic pavement detection systems are only used for solving some problems, and the problems generated by roads cannot be solved comprehensively, quickly and efficiently. For example, the scattered objects on the highway depend on two persons driving a engineering vehicle, and 13.6 kilometers of highway mileage in China are visually checked along the highway every day by 50 kilometers, so that a great deal of manpower and material resources are consumed.
Disclosure of Invention
In order to solve the defects of the prior art, the embodiment of the disclosure provides an unmanned aerial vehicle-mounted road surface detection system, which can dynamically detect the road surface in real time.
In order to achieve the purpose, the following technical scheme is adopted in the application:
the unmanned aerial vehicle-mounted road surface detection system comprises an unmanned aerial vehicle, wherein a GPS system and an unmanned aerial vehicle flight height control system are arranged on the unmanned aerial vehicle;
the unmanned aerial vehicle is provided with a camera, and the camera is connected with an image sensor; the output end of the image sensor is connected to the input end of the signal amplification circuit, and the output end of the signal amplification circuit is connected to the input end of the signal conversion circuit; the output end of the signal conversion circuit is connected to a wireless data transmission station;
the input end of the signal amplification circuit is connected with the output end of the signal detection and processing circuit, and the input end of the signal detection and processing circuit is connected with the controller;
the ground wireless data transmission radio station is connected with the ground server through the level adapter plate, and the wireless data transmission radio station carried on the unmanned aerial vehicle is communicated with the ground wireless data transmission radio station in a wireless mode.
The ground server is used for processing the acquired image, including graying, image smoothing and sharpening, image gray level conversion and image segmentation;
extracting corresponding object characteristics from the image after image processing;
and respectively carrying out image processing on different shot images to obtain image data sets, carrying out classification processing on the collected data sets, and carrying out pre-training in a convolution network by using the image data sets to optimize the model.
The further technical scheme is that pre-training is carried out in a convolutional network: carrying out back propagation iteration on each image characteristic parameter in the data set by adopting a back propagation algorithm and a random gradient descent method according to the magnitude of a loss value of forward propagation to update the weight of each layer, stopping training the model until the loss value of the model tends to be converged to obtain a deep learning model, and extracting deep learning characteristics from a penultimate layer of full convolution layer of the image;
and inputting any given image to be recognized into a trained deep learning model, extracting the deep learning characteristics of the sample, and judging the category of the image.
According to a further technical scheme, the image types included in the image data set include but are not limited to a pavement subsidence image, a pavement fracture image, a scattered object image, a solid waste image generated in construction and a small animal image staying in the middle of a road.
The embodiment of the disclosure also discloses a detection method of the unmanned aerial vehicle-mounted road surface detection system, which comprises the following steps:
positioning the position of the unmanned aerial vehicle, judging the road detected by the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to fly along the detected road;
the unmanned aerial vehicle flight height control system controls the output power of the unmanned aerial vehicle according to the electric wave emitted vertically downwards and the electric wave emitted from the ground so as to adjust the flight height of the unmanned aerial vehicle;
according to the detected road surface where the unmanned aerial vehicle is located, the camera shoots images, so that the shot road images can be continuously butted front and back, the images are processed and then sent to the controller, the controller sends image digital signals to the ground wireless data transmission radio station through the wireless data transmission radio station, and the image digital signals are transmitted to the ground server through the ground wireless data transmission radio station;
processing images in a ground server, wherein the processing comprises graying, image smoothing and sharpening, image gray level transformation and image segmentation;
extracting corresponding object characteristics from the image after image processing;
and respectively carrying out image processing on different shot images to obtain image data sets, carrying out classification processing on the collected data sets, and carrying out pre-training in a convolutional network by using a self-made data set to optimize the model.
The further technical scheme is that pre-training is carried out in a convolutional network: carrying out back propagation iteration on each image characteristic parameter in the data set by adopting a back propagation algorithm and a random gradient descent method according to the magnitude of a loss value of forward propagation to update the weight of each layer, stopping training the model until the loss value of the model tends to be converged to obtain a deep learning model, and extracting deep learning characteristics from a penultimate layer of full convolution layer of the image;
and inputting any given image to be recognized into a trained deep learning model, extracting the deep learning characteristics of the sample, and judging the category of the image.
According to a further technical scheme, the image types included in the image data set include but are not limited to a pavement subsidence image, a pavement fracture image, a scattered object image, a solid waste image generated in construction and a small animal image staying in the middle of a road.
The further technical scheme is that the graying specifically comprises the following steps:
the image collected by the camera to the computer is in RGB format, and in the converted gray image, one pixel represents the gray value thereof, according to the formula:
Y＝0.299R+0.587G+0.114B
r, G, B for each point the pixel values red, green and blue respectively range from 0 to 255, thus R, G, B is assigned to Y, and the pixel for each point in the image has only one value.
According to the further technical scheme, after gray processing, gray conversion is carried out. Let Y be the gray before transformation and S be the gray after transformation, the general formula for logarithmic transformation is:
S＝clog(1+Y)
wherein c is a constant, Y is more than or equal to 0, the low gray value with narrow range in the source image is mapped to the gray interval with wider range by logarithmic transformation, and the high gray value interval with wider range is mapped to the narrow gray interval.
Further technical scheme, image smoothing: and (4) delineating a region in the same image, and processing the image by adopting a neighborhood averaging method. The principle is to add the gray value of each pixel in the original image and the gray values of its neighboring 8 pixels, and then to use the obtained average value (divided by 9) as the gray value of the pixel in the new image. Namely:
m is the coordinates of each neighboring pixel in the taken neighborhood, and N is the number of neighboring pixels contained in the neighborhood.
The further technical scheme is that the image sharpening process comprises the following steps: with the linear sharpening process, a linear high-pass filter is the most common linear sharpening filter and can be implemented in MATLAB by calling the filter2 function and the fspecial function.
The further technical scheme is that the image segmentation: the image segmentation method based on edge detection firstly determines edge pixels in an image, and then connects the pixels together to form a required area boundary.
According to the further technical scheme, corresponding object features are extracted from the image after image processing, and the image features comprise color features, texture features, shape features and space features;
color characteristics: extracting the color characteristics of the object by adopting a color moment method, wherein the color distribution information is mainly concentrated in a low-order moment, and the first-order moment, the second-order moment and the third-order moment of the color can be adopted to express the color characteristics of the object;
texture characteristics: the image texture features are characteristic quantization for extracting gray level change in an image region, and the texture features are extracted from an autocorrelation function of the image, namely an energy spectrum function of the image by adopting a statistical method, namely the thickness and the directional characteristic parameters of the texture are extracted by calculating the energy spectrum function of the image.
Shape characteristics: the Fourier shape descriptor method uses Fourier transformation of object boundary as shape description, converts two-dimensional problem into one-dimensional problem by using closure and periodicity of region boundary, and derives three shape expressions from boundary point, which are curvature function, centroid distance and complex coordinate function.
Spatial characteristics: the segmented image is divided into object or color regions contained in the image, and then image features are extracted from these regions and an index is established.
Compared with the prior art, the beneficial effect of this disclosure is:
the technical scheme can collect different road surface information, respectively carry out image processing, establish corresponding models and dynamically detect the road surface in real time.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.
FIG. 1 is a flow diagram illustrating detection in accordance with one or more embodiments of the present disclosure;
FIG. 2 is a block diagram of a detection system in accordance with one or more embodiments of the present disclosure.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
In a typical implementation of this application, as shown in fig. 2, a road surface detecting system that unmanned aerial vehicle carried is provided, including unmanned aerial vehicle, the GPS system, unmanned aerial vehicle flying height control system, unmanned aerial vehicle is last to be loaded with the camera, image sensor, controller, two wireless data transfer radio stations (one is carried by unmanned aerial vehicle, and another connects the computer), level keysets, computer. The camera is connected with the image sensor; a signal amplifying circuit and a signal converting circuit are sequentially connected between the image sensor and the controller; the output of the image sensor is connected with the input of the signal amplifying circuit, and the output of the signal amplifying circuit is connected with the input of the signal converting circuit; the output of the signal conversion circuit is connected with a wireless data transmission radio station. And a signal detection and processing circuit is connected between the signal amplification circuit and the controller, namely, the input of the signal amplification circuit is connected with the output of the signal detection and processing circuit, and the input of the signal detection and processing circuit is connected with the controller. The ground wireless data transmission radio station is connected with a computer through a level adapter plate.
In a typical embodiment of the present application, as shown in fig. 1, an unmanned aerial vehicle-mounted road surface detection method is disclosed, which includes the following specific operation steps:
the method comprises the following steps: unmanned aerial vehicle's control system
The unmanned aerial vehicle control system comprises a GPS system and an unmanned aerial vehicle flying height control system.
The adoption of GPS to realize unmanned aerial vehicle autonomous navigation is that the unmanned aerial vehicle warp, latitude, time and other information provided by an airborne Global Positioning System (GPS) receiver is used as feedback data of planned route information, and is combined with other parameters to be jointly used as control parameters, and the unmanned aerial vehicle elevator and the aileron rudder are mainly controlled to change the flight attitude of the unmanned aerial vehicle, so that the unmanned aerial vehicle autonomously flies according to the planned route
The unmanned aerial vehicle flight height control system is designed based on the height of an unmanned aerial vehicle controlled by PI, realizes the height control of the unmanned aerial vehicle by using a proportional plus integral control method, and adopts a layer-by-layer design method from an inner loop to an outer loop; a loop separation method is used for analyzing the closed loop transfer function to determine an approximate mathematical model of the height control system. And according to a time domain method and a frequency domain method, repeatedly comparing the time domain response indexes and the frequency domain response indexes of all the groups of parameters in a value range until a group of optimal parameter values is obtained, and controlling the output power of the unmanned aerial vehicle by the control system according to the electric waves emitted vertically downwards and the electric waves emitted from the ground so as to adjust the flying height of the unmanned aerial vehicle.
Step two: acquiring an image:
according to the detection road surface where the unmanned aerial vehicle is located, the camera shoots images, the time interval of the unmanned aerial vehicle shooting images is controlled, the shot road images can be continuously butted front and back, the images are sent to the image sensor, the image sensor converts the images into electronic signals and transmits the electronic signals to the signal amplification circuit, the electronic signals are amplified by the signal amplification circuit and transmitted to the signal conversion circuit, the analog signals are converted into digital signals by the signal conversion circuit and transmitted to the controller, the controller detects and processes the electronic signals through the signal detection and processing circuit and feeds the electronic signals back to the signal amplification circuit, the signals are finally stable, the controller sends the stable image digital signals to the ground wireless data transmission radio station through the wireless data transmission radio station, and the stable image digital signals are transmitted to the computer through the ground wireless data transmission radio station. The controller may also store the image signal to the memory.
Step three: image processing: the image processing is performed on the ground. Image processing generally comprises four steps: (1) graying, (2) image grayscale conversion, (3) image sharpening and smoothing, (4) image segmentation, and the like. The image sharpening and smoothing can be carried out in the same image in a partitioning mode, and one of the methods can be adopted for different images.
1) Graying
The image collected by the camera to the computer is in RGB format, and in the converted gray image, one pixel represents the gray value thereof, according to the formula:
Y＝0.299R+0.587G+0.114B
r, G, B for each point the pixel values red, green and blue respectively range from 0 to 255, thus R, G, B is assigned to Y, and the pixel for each point in the image has only one value.
2) Image grey scale transformation
After the gradation processing, gradation conversion is performed. Let Y be the gray before transformation and S be the gray after transformation, and the general formula of logarithmic transformation by logarithmic transformation is:
S＝clog(1+Y)
wherein c is a constant, Y is more than or equal to 0, logarithmic transformation maps the low gray value with narrow range in the source image to the gray interval with wide range, and maps the high gray value interval with wide range to the narrow gray interval, thereby expanding the value of dark pixel, compressing the value of high gray, and enhancing the low gray detail in the image.
3) Image smoothing
And (4) delineating a region in the same image, and processing the image by adopting a neighborhood averaging method. The principle is to add the gray value of each pixel in the original image and the gray values of its neighboring 8 pixels, and then to use the obtained average value (divided by 9) as the gray value of the pixel in the new image. Namely:
m is the coordinates of each neighboring pixel in the taken neighborhood, and N is the number of neighboring pixels contained in the neighborhood.
4) Image sharpening process
With the linear sharpening process, a linear high-pass filter is the most common linear sharpening filter and can be implemented in MATLAB by calling the filter2 function and the fspecial function.
5) Image segmentation
The image segmentation method based on edge detection firstly determines edge pixels in an image, and then connects the pixels together to form a required area boundary.
Step four: extracting characteristic parameters
And extracting different object characteristics according to different conditions of the road surface. Common image features include color features, texture features, shape features, spatial features, and the like.
1) Color characteristics
The color characteristics of the object are extracted by adopting a color moment method, the color distribution information is mainly concentrated in a low-order moment, and the color characteristics of the object can be expressed by adopting a first-order moment, a second-order moment and a third-order moment of the color.
2) Texture features
The image texture features are feature quantization for extracting gray level change in an image region, and the texture features are extracted from an autocorrelation function (namely an energy spectrum function of the image) of the image by adopting a statistical method, namely, feature parameters such as the thickness and the directionality of the texture are extracted by calculating the energy spectrum function of the image.
3) Shape feature
The fourier shape descriptor method uses the fourier transform of the object boundary as the shape description, and converts the two-dimensional problem into the one-dimensional problem by using the closeness and periodicity of the region boundary. Three shape expressions are derived from the boundary points, namely a curvature function, a centroid distance and a complex coordinate function.
4) Spatial characteristics
The segmented image is divided into object or color regions contained in the image, and then image features are extracted from these regions and an index is established.
Step five: modeling
1) Constructing a self-made data set
And acquiring images through various conditions possibly occurring on the road surface, and acquiring a large number of object data sets (characteristic parameters) through the acquired images in the third step and the fourth step to prepare for establishing the model.
2) Modeling
And extracting the acquired data set, classifying the acquired data set, and processing the acquired data set to obtain various conditions of the road surface. An unmanned airborne road surface detection system is written by utilizing OpenCV and Visual Studio.
The system comprises various conditions of the road surface, such as subsidence of the road surface, breakage of the road surface, scattering of objects, untimely cleaning of solid wastes generated in construction, small animals staying in the middle of the road and the like. Training a convolutional neural network: and (3) carrying out back propagation iteration on each image characteristic parameter in the data set by adopting a back propagation algorithm and a random gradient descent method according to the magnitude of the forward propagation loss value to update the weight of each layer, stopping training the model until the loss value of the model tends to be converged to obtain a deep learning model, and extracting the deep learning characteristic from the penultimate full convolution layer of the image. Given any image to be recognized, inputting the image into a trained deep learning model, extracting the deep learning characteristics of a sample, and effectively judging which category the image belongs to through a training method.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (8)
1. The detection method of the unmanned aerial vehicle-mounted road surface detection system is characterized in that,
positioning the position of the unmanned aerial vehicle, judging the road detected by the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to fly along the detected road;
the unmanned aerial vehicle flight height control system controls the output power of the unmanned aerial vehicle according to the electric wave emitted vertically downwards and the electric wave emitted from the ground so as to adjust the flight height of the unmanned aerial vehicle;
according to the detected road surface where the unmanned aerial vehicle is located, the camera shoots images, the time interval of the unmanned aerial vehicle shooting images is controlled, the shot road images can be continuously butted front and back, the images are processed and then sent to the controller, the controller sends image digital signals to the ground wireless data transmission radio station through the wireless data transmission radio station, and the image digital signals are transmitted to the ground server through the ground wireless data transmission radio station;
processing images in a ground server, wherein the processing comprises graying, image smoothing and sharpening, image gray level transformation and image segmentation;
respectively carrying out image processing on different shot images to obtain image data sets, carrying out classification processing on the collected data sets, using a self-made data set to carry out pre-training in a convolution network, and optimizing the model;
the method can acquire different pavement information, respectively perform image processing, establish corresponding models and dynamically detect the pavement in real time; the image data set comprises image types including but not limited to a pavement subsidence image, a pavement fracture image, a scattered object image, a solid waste generated in construction, an untimely cleaning image and a small animal image staying in the middle of a road;
extracting corresponding object features from the image after image processing, wherein the image features comprise color features, texture features, shape features and space features;
color characteristics: extracting the color characteristics of the object by adopting a color moment method, wherein the color distribution information is mainly concentrated in a low-order moment, and the first-order moment, the second-order moment and the third-order moment of the color can be adopted to express the color characteristics of the object;
texture characteristics: the image texture features are characteristic quantization for extracting gray level change in an image region, and the texture features are extracted from an autocorrelation function of the image, namely an energy spectrum function of the image by adopting a statistical method, namely the thickness and the directional characteristic parameters of the texture are extracted by calculating the energy spectrum function of the image;
shape characteristics: the Fourier shape descriptor method uses Fourier transformation of an object boundary as shape description, converts a two-dimensional problem into a one-dimensional problem by using the closure and periodicity of a region boundary, and derives three shape expressions which are respectively a curvature function, a centroid distance and a complex coordinate function from a boundary point;
spatial characteristics: the segmented image is divided into object or color regions contained in the image, and then image features are extracted from these regions and an index is established.
2. The method of claim 1, wherein the pre-training is performed in a convolutional network: and (3) carrying out back propagation iteration on each image characteristic parameter in the data set by adopting a back propagation algorithm and a random gradient descent method according to the magnitude of the forward propagation loss value to update the weight of each layer, stopping training the model until the loss value of the model tends to be converged to obtain a deep learning model, and extracting the deep learning characteristic from the penultimate full convolution layer of the image.
3. The method of claim 1, wherein the graying is specifically:
the image collected by the camera to the computer is in RGB format, and in the converted gray image, one pixel represents the gray value thereof, according to the formula:
Y＝0.299R+0.587G+0.114B
r, G, B for each point the pixel values red, green, and blue, respectively, range from 0 to 255, thus assigning R, G, B to Y, the pixel for each point in the image has only one value;
after the gray scale processing, gray scale conversion is carried out, Y is the gray scale before conversion, S is the gray scale after conversion, and the general formula adopting logarithmic conversion is as follows:
S＝clog(1+Y)
wherein c is a constant, Y is more than or equal to 0, the low gray value with narrow range in the source image is mapped to the gray interval with wide range by logarithmic transformation, and the high gray value interval with wide range is mapped to the narrow gray interval.
4. The method of claim 1, wherein the image smoothing is: a region is selected from the same image, and the image is processed by adopting a neighborhood averaging method; the principle is that the gray value of each pixel in the original image is added with the gray values of 8 adjacent pixels around the pixel, and then the obtained average value is used as the gray value of the pixel in a new image;
image sharpening processing: with the linear sharpening process, a linear high-pass filter is the most commonly used linear sharpening filter, and can be implemented in MATLAB by calling the filter2 function and the fspecial function.
5. The method of claim 1, wherein the image segmentation comprises: the image segmentation method based on edge detection firstly determines edge pixels in an image, and then connects the pixels together to form a required area boundary.
6. The detection method of the unmanned aerial vehicle-mounted road surface detection system according to claim 1, wherein the unmanned aerial vehicle-mounted road surface detection system comprises an unmanned aerial vehicle, and a GPS system and an unmanned aerial vehicle flight height control system are arranged on the unmanned aerial vehicle;
the unmanned aerial vehicle is provided with a camera, and the camera is connected with an image sensor; the output end of the image sensor is connected to the input end of the signal amplification circuit, and the output end of the signal amplification circuit is connected to the input end of the signal conversion circuit; the output end of the signal conversion circuit is connected to a wireless data transmission station;
the input end of the signal amplification circuit is connected with the output end of the signal detection and processing circuit, and the input end of the signal detection and processing circuit is connected with the controller;
the ground wireless data transmission radio station is connected with the ground server through a level adapter plate, and the wireless data transmission radio station carried on the unmanned aerial vehicle and the ground wireless data transmission radio station communicate in a wireless mode;
the ground server is used for processing the acquired image, including graying, image smoothing and sharpening, image gray level transformation and image segmentation;
extracting corresponding object characteristics from the image after image processing;
respectively carrying out image processing on different shot images to obtain image data sets, carrying out classification processing on the collected data sets, carrying out pre-training in a convolution network by using the image data sets, and optimizing the model;
the image data set comprises image types including images of scattered objects, images of solid wastes generated in construction, images of small animals staying in the middle of a road, wherein the images are not cleaned in time;
pre-training in a convolutional network: and (3) carrying out back propagation iteration on each image characteristic parameter in the data set by adopting a back propagation algorithm and a random gradient descent method according to the magnitude of the forward propagation loss value to update the weight of each layer, stopping training the model until the loss value of the model tends to be converged to obtain a deep learning model, and extracting the deep learning characteristic from the penultimate full convolution layer of the image.
7. The method of claim 6, wherein the step of detecting the position of the vehicle is performed by a robot,
and inputting any given image to be recognized into a trained deep learning model, extracting the deep learning characteristics of the sample, and judging the category of the image.
8. The method of claim 6, wherein the image data set further includes image types including but not limited to a road surface sag image, a road surface fracture image.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2018116146425 | 2018-12-27 | ||
| CN201811614642.5A CN109447047A (en) | 2018-12-27 | 2018-12-27 | The pavement detection system and detection method of UAV system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109902668A CN109902668A (en) | 2019-06-18 |
| CN109902668B true CN109902668B (en) | 2022-04-22 |
Family
ID=65539655
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811614642.5A Withdrawn CN109447047A (en) | 2018-12-27 | 2018-12-27 | The pavement detection system and detection method of UAV system |
| CN201910305803.0A Active CN109902668B (en) | 2018-12-27 | 2019-04-16 | Unmanned aerial vehicle-mounted road surface detection system and detection method |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811614642.5A Withdrawn CN109447047A (en) | 2018-12-27 | 2018-12-27 | The pavement detection system and detection method of UAV system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN109447047A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112666963A (en) * | 2020-12-18 | 2021-04-16 | 浙江师范大学 | Road pavement crack detection system based on four-axis unmanned aerial vehicle and detection method thereof |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7840059B2 (en) * | 2006-09-21 | 2010-11-23 | Microsoft Corporation | Object recognition using textons and shape filters |
| WO2013090830A1 (en) * | 2011-12-16 | 2013-06-20 | University Of Southern California | Autonomous pavement condition assessment |
| CN104331873A (en) * | 2013-07-22 | 2015-02-04 | 浙江大学 | Method for detecting road from single image |
| CN205003549U (en) * | 2015-08-24 | 2016-01-27 | 深圳市诺亚星辰科技开发有限公司 | Single rotor unmanned aerial vehicle is flight control hardware systems independently |
| CN105389988B (en) * | 2015-12-07 | 2018-03-06 | 北京航空航天大学 | A kind of express highway intelligent cruising inspection system of multiple no-manned plane collaboration |
| CN106012780B (en) * | 2016-07-15 | 2019-01-29 | 京东方科技集团股份有限公司 | Pavement detection equipment and pavement detection method |
| CN106651872B (en) * | 2016-11-23 | 2020-09-15 | 北京理工大学 | Pavement crack identification method and system based on Prewitt operator |
| CN207399371U (en) * | 2017-10-31 | 2018-05-22 | 南京开天眼无人机科技有限公司 | A kind of Unmanned Aerial Vehicle Data overlapping system |
| CN107945153A (en) * | 2017-11-07 | 2018-04-20 | 广东广业开元科技有限公司 | A kind of road surface crack detection method based on deep learning |
| CN108416307B (en) * | 2018-03-13 | 2020-08-14 | 北京理工大学 | Method, device and equipment for detecting pavement cracks of aerial images |
-
2018
- 2018-12-27 CN CN201811614642.5A patent/CN109447047A/en not_active Withdrawn
-
2019
- 2019-04-16 CN CN201910305803.0A patent/CN109902668B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109447047A (en) | 2019-03-08 |
| CN109902668A (en) | 2019-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101839599B1 (en) | Road facility surveying system using drone | |
| CN107316064B (en) | Asphalt pavement crack classification and identification method based on convolutional neural network | |
| CN111985274B (en) | Remote sensing image segmentation method based on convolutional neural network | |
| CN105550692B (en) | The homing vector landing concept of unmanned plane based on marker color and contour detecting | |
| CN105318888A (en) | Unmanned perception based unmanned aerial vehicle route planning method | |
| CN110770791A (en) | Image boundary acquisition method and device based on point cloud map and aircraft | |
| CN114089786A (en) | Autonomous inspection system based on unmanned aerial vehicle vision and along mountain highway | |
| CN110799983A (en) | Map generation method, map generation equipment, aircraft and storage medium | |
| CN111339876B (en) | Method and device for identifying types of areas in scene | |
| Ruban et al. | Method for determining elements of urban infrastructure objects based on the results from air monitoring | |
| CN114038193B (en) | Intelligent traffic flow data statistics method and system based on unmanned aerial vehicle and multi-target tracking | |
| CN109902668B (en) | Unmanned aerial vehicle-mounted road surface detection system and detection method | |
| CN111458691B (en) | Building information extraction method and device and computer equipment | |
| Xu et al. | Development of power transmission line detection technology based on unmanned aerial vehicle image vision | |
| CN110986884A (en) | Unmanned aerial vehicle-based aerial survey data preprocessing and vegetation rapid identification method | |
| CN104735425B (en) | Based on the transmission facility identification method of unmanned plane | |
| CN110033045A (en) | A kind of method and apparatus of trained identification image atomization | |
| CN104537623A (en) | Image fog-removing method and device based on image segmentation | |
| CN112926415A (en) | Pedestrian avoiding system and pedestrian monitoring method | |
| CN108288388A (en) | A kind of intelligent traffic monitoring system | |
| CN104700428B (en) | A kind of desert areas Approach for road detection be positioned on unmanned plane | |
| CN109446955B (en) | Image processing method and device, unmanned aerial vehicle and server | |
| CN116030065A (en) | Road quality detection method based on image recognition | |
| CN104715462A (en) | Desert region road detection platform on unmanned aerial vehicle | |
| CN108737775A (en) | Big data acquisition platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |