CN116994160A - Slope disease detection method, device and system based on unmanned aerial vehicle acquired image - Google Patents
Slope disease detection method, device and system based on unmanned aerial vehicle acquired image Download PDFInfo
- Publication number
- CN116994160A CN116994160A CN202310960856.2A CN202310960856A CN116994160A CN 116994160 A CN116994160 A CN 116994160A CN 202310960856 A CN202310960856 A CN 202310960856A CN 116994160 A CN116994160 A CN 116994160A
- Authority
- CN
- China
- Prior art keywords
- image
- slope
- aerial vehicle
- unmanned aerial
- disease detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 88
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 88
- 238000001514 detection method Methods 0.000 title claims abstract description 81
- 238000012545 processing Methods 0.000 claims abstract description 41
- 238000006073 displacement reaction Methods 0.000 claims abstract description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 44
- 238000000034 method Methods 0.000 claims description 27
- 238000001914 filtration Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000003672 processing method Methods 0.000 claims description 4
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 238000007689 inspection Methods 0.000 abstract description 12
- 238000012549 training Methods 0.000 description 42
- 238000012216 screening Methods 0.000 description 37
- 238000011176 pooling Methods 0.000 description 19
- 230000004927 fusion Effects 0.000 description 15
- 230000006870 function Effects 0.000 description 10
- 238000012360 testing method Methods 0.000 description 10
- 230000006835 compression Effects 0.000 description 9
- 238000007906 compression Methods 0.000 description 9
- 238000000605 extraction Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 239000013598 vector Substances 0.000 description 7
- 230000004913 activation Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 5
- 238000003708 edge detection Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000001629 suppression Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007670 refining Methods 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000010224 classification analysis Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Remote Sensing (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The application relates to the technical field of image processing, in particular to a slope disease detection method, device and system based on unmanned aerial vehicle collected images. Image acquisition is carried out on the slope area through the unmanned aerial vehicle, and an unmanned aerial vehicle acquired image is obtained; performing template matching on the image acquired by the unmanned aerial vehicle, and identifying a slope image; carrying out differential processing on the slope image to obtain a slope characteristic image; slope disease detection is carried out according to the slope characteristic images; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch. According to the scheme, the unmanned aerial vehicle is used for rapidly acquiring the image of the slope area and nearby processing the acquired data to obtain the slope image, so that the manual inspection workload is reduced, and compared with the mode of manually identifying and detecting the slope disease, the slope image is rapidly identified by using template matching, further the slope characteristic image is acquired, the slope crack, the local displacement of the slope and the blockage of the drainage ditch are detected according to the slope characteristic image, and the automatic detection of the slope disease is realized.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a slope disease detection method, device and system based on unmanned aerial vehicle collected images.
Background
With the continuous advancement of urban areas, more and more areas are being built on roads, and the construction of slopes is inevitably involved. The side slope is an earthquake disaster prone area, and as the side slope is often affected by factors such as earthquake, rainwater and the like, once the side slope collapses, serious threat is caused to lives and properties of buildings and people. Therefore, slope monitoring is an indispensable ring in road construction, and the loss can be greatly reduced by timely detecting the slope diseases.
At present, a manual inspection mode is generally adopted for slope detection. The traditional manual inspection adopts the modes of visual inspection, knocking, touching and the like, wherein the visual inspection is mainly performed, and the manual recording mode is generally adopted for recording inspection results. The traditional manual inspection has potential safety hazards in the inspection of the side slope, and has the defects of lack of image data, inspection blind areas, lack of special database storage, system management and the like. To this problem, the prior art adopts unmanned aerial vehicle automatic cruise shooting's mode to solve. However, at present, the automatic cruising of the unmanned aerial vehicle can only acquire cruising images of a slope area, and technicians are still required to process the cruising images so as to judge whether the slope has disease conditions or not.
Disclosure of Invention
Aiming at the defects in the prior art, the application provides a slope disease detection method, device and system based on unmanned aerial vehicle collected images, so as to realize automatic detection of the slope disease, thereby reducing the manual workload.
In a first aspect, the application provides a slope disease detection method based on unmanned aerial vehicle collected images.
In a first implementation manner, a slope disease detection method based on unmanned aerial vehicle collected images includes:
image acquisition is carried out on the slope area through the unmanned aerial vehicle, and an unmanned aerial vehicle acquired image is obtained;
performing template matching on the image acquired by the unmanned aerial vehicle, and identifying a slope image;
carrying out differential processing on the slope image to obtain a slope characteristic image;
slope disease detection is carried out according to the slope characteristic images; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
In combination with the first implementation manner, in a second implementation manner, before performing template matching on the image acquired by the unmanned aerial vehicle, the method further includes:
and carrying out image enhancement processing on the image acquired by the unmanned aerial vehicle.
In combination with the first implementation manner, in a third implementation manner, performing template matching on the image acquired by the unmanned aerial vehicle, and identifying the slope image includes:
constructing an image pyramid according to the image acquired by the unmanned aerial vehicle;
selecting a target template, and carrying out a template matching algorithm on each layer of image pyramid based on the target template to obtain the optimal matching position of each layer of image pyramid;
obtaining an optimal matching result according to the optimal matching position of each layer of image pyramid;
and judging whether the slope image is a slope image or not according to the best matching result.
In combination with the third implementation manner, in a fourth implementation manner, the template matching algorithm includes a violent matching algorithm, a normalized cross-correlation matching algorithm, a correlation coefficient matching algorithm or a square difference matching algorithm.
In combination with the first implementation manner, in a fifth implementation manner, performing differential processing on the slope image to obtain a slope feature image, where the differential processing includes:
preprocessing a side slope image to obtain a side slope image to be detected;
and carrying out differential processing on the slope image to be detected by adopting an inter-frame differential algorithm to obtain a slope characteristic image.
With reference to the fifth implementation manner, in a sixth implementation manner, preprocessing the slope image to obtain a slope image to be detected includes:
denoising the slope image by adopting Gaussian filtering and mean square filtering to obtain a denoised image;
image enhancement is carried out on the reflection component of the denoising image by adopting an image enhancement method based on guide filtering, so as to obtain an enhanced image;
and carrying out edge refinement on the enhanced image by adopting a non-maximum value inhibition and self-adaptive threshold processing method to obtain the slope image to be detected.
With reference to the first implementation manner, in a seventh implementation manner, the slope disease detection according to the slope feature image includes:
and inputting the slope characteristic image into a preset slope disease detection model to obtain a slope disease detection result.
In a second aspect, the application provides a slope disease detection device based on unmanned aerial vehicle collected images.
In an eighth implementation manner, a slope disease detection device based on image acquisition of an unmanned aerial vehicle includes:
the unmanned aerial vehicle acquisition image acquisition module is configured to acquire images of the slope area through the unmanned aerial vehicle, so as to acquire unmanned aerial vehicle acquisition images;
the side slope image recognition module is configured to perform template matching on the collected images of the unmanned aerial vehicle and recognize side slope images;
the slope characteristic image acquisition module is configured to perform differential processing on the slope image to acquire a slope characteristic image;
the slope disease detection module is configured to detect slope disease according to the slope characteristic image; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
In a third aspect, the application provides a slope disease detection system based on unmanned aerial vehicle collected images.
In a ninth implementation manner, a slope disease detection system based on image acquisition of an unmanned aerial vehicle includes: unmanned aerial vehicle, unmanned aerial vehicle embedded equipment and cloud server of carrying surveillance camera head: the unmanned aerial vehicle acquires images of the side slope area through the camera, acquires the images acquired by the unmanned aerial vehicle and transmits the images to the unmanned aerial vehicle embedded equipment; the unmanned aerial vehicle embedded device processes the image acquired by the unmanned aerial vehicle to obtain a side slope image, detects side slope diseases according to the side slope image, and sends the side slope image and the side slope disease detection result to the remote server.
According to the technical scheme, the beneficial technical effects of the application are as follows:
1. according to the scheme, the unmanned aerial vehicle is used for rapidly acquiring the image of the slope area and nearby processing the acquired data to obtain the slope image, so that the manual inspection workload is reduced. And compare the mode of adopting artifical discernment to detect the side slope disease after gathering the picture, this scheme utilizes through template matching, and quick discernment side slope image, and then acquire side slope characteristic image, detect side slope crack, side slope local displacement and escape canal jam according to side slope characteristic image, realized the automated inspection of side slope disease, the response time of testing result shortens greatly to can in time carry out the side slope early warning, improve side slope safety, and realized the wholeness and the instantaneity of data acquisition, side slope image recognition and side slope disease detection.
2. The unmanned aerial vehicle collects images of the slope area, and the unmanned aerial vehicle detects the slope diseases in real time, so that the data transmission process is reduced, the manual work load is reduced, the response time of the slope disease detection result is improved, and the safety of the slope is further guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. Like elements or portions are generally identified by like reference numerals throughout the several figures. In the drawings, elements or portions thereof are not necessarily drawn to scale.
Fig. 1 is a schematic diagram of a slope disease detection method based on an image collected by an unmanned aerial vehicle according to the embodiment;
fig. 2 is a schematic structural diagram of a slope image recognition model according to the present embodiment;
fig. 3 is a schematic structural diagram of a channel feature screening submodule provided in the present embodiment;
fig. 4 is a schematic structural diagram of a spatial feature screening submodule according to the present embodiment;
fig. 5 is a schematic structural diagram of a model for detecting a side slope disease according to the present embodiment;
fig. 6 is a schematic structural diagram of a slope disease detection system based on an image collected by an unmanned aerial vehicle according to the present embodiment;
fig. 7 is a schematic diagram of another slope disease detection method based on an image collected by an unmanned aerial vehicle according to the present embodiment;
fig. 8 is a schematic structural diagram of a slope disease detection device based on an image collected by an unmanned aerial vehicle according to the present embodiment;
the accompanying drawings:
1-unmanned aerial vehicle, 2-camera, 3-unmanned aerial vehicle embedded equipment, 4-high in the clouds server.
Detailed Description
Embodiments of the technical scheme of the present application will be described in detail below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present application, and thus are merely examples, and are not intended to limit the scope of the present application.
It is noted that unless otherwise indicated, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms first, second and the like in the description and in the claims of the embodiments of the disclosure and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to implement the embodiments of the disclosure described herein. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. The term "plurality" means two or more, unless otherwise indicated. In the embodiment of the present disclosure, the character "/" indicates that the front and rear objects are an or relationship. For example, A/B represents: a or B. The term "and/or" is an associative relationship that describes an object, meaning that there may be three relationships. For example, a and/or B, represent: a or B, or, A and B. The term "corresponding" may refer to an association or binding relationship, and the correspondence between a and B refers to an association or binding relationship between a and B.
Referring to fig. 1, a slope disease detection method based on unmanned aerial vehicle collected images includes:
s11, acquiring images of a slope area through an unmanned aerial vehicle to obtain unmanned aerial vehicle acquired images;
step S12, performing template matching on the image acquired by the unmanned aerial vehicle, and identifying a slope image;
s13, carrying out differential processing on the slope image to obtain a slope characteristic image;
s14, detecting slope diseases according to the slope characteristic images; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
Optionally, before performing template matching on the image acquired by the unmanned aerial vehicle, the method further includes: and carrying out image enhancement processing on the image acquired by the unmanned aerial vehicle.
In some embodiments, performing image enhancement processing on the unmanned aerial vehicle acquired image includes: adding noise, rotating translation and manually simulating diseases.
Optionally, performing template matching on the unmanned aerial vehicle acquired image, and identifying the slope image includes: constructing an image pyramid according to the image acquired by the unmanned aerial vehicle; selecting a target template, and carrying out a template matching algorithm on each layer of image pyramid based on the target template to obtain the optimal matching position of each layer of image pyramid; obtaining an optimal matching result according to the optimal matching position of each layer of image pyramid; and judging whether the slope image is a slope image or not according to the best matching result.
In some embodiments, a Pyramid Sampling (Pyramid Sampling) template matching algorithm is a multi-scale template matching method for finding the location of a target template in an image. The template matching is carried out under different scales by constructing an image pyramid so as to adapt to the scale change of the target template.
In some embodiments, the detailed steps of the pyramid sampling template matching algorithm include:
(1) Constructing an image pyramid: first, pyramid decomposition is performed on the images to be matched to generate a series of images with different scales, and the image pyramid is usually generated by methods such as downsampling (resolution reduction) or Gaussian blur.
(2) Defining a target template: a small target template is selected as the object or pattern to be found.
(3) For each pyramid layer:
a. performing a template matching algorithm, such as a squared difference template matching or a correlation coefficient template matching, on the current pyramid layer may select an appropriate algorithm according to specific needs.
b. And determining the best matching position on the current pyramid layer according to the matching result.
c. And mapping the best matching position into an original image coordinate system according to the scale relation of the pyramid.
(4) Repeating the step (3) on different pyramid layers until all pyramid layers are traversed.
(5) Obtaining a final matching result: and selecting the optimal matching position as a final result according to the matching results on all pyramid layers.
The pyramid sampling template matching algorithm can effectively cope with the scale change of the target template, and the matching position most suitable for the scale of the target template can be found by matching on images with different scales. In addition, pyramid sampling can also improve the robustness of the algorithm, so that the algorithm has certain tolerance to factors such as illumination change, rotation and the like.
Optionally, the template matching algorithm comprises a brute force matching algorithm, a normalized cross-correlation matching algorithm, a correlation coefficient matching algorithm, or a squared difference matching algorithm.
In some embodiments, the violent matching algorithm is a pixel-based template matching algorithm. The basic principle is that each pixel point in the target image is traversed, compared with the template image and the most similar position is found out. And comparing the pixel points of the template image and the target image. For each pixel point in the template image, the corresponding pixel point is found in the target image, and the error between them is calculated. The error may be expressed using a distance between pixel points, euclidean distance, square error, or the like. Then, the position with the smallest error is found and marked as the best matching position. Finally, the best matching position is marked in the target image, or a measurement value of the matching degree is calculated.
In some embodiments, a normalized cross-correlation matching algorithm (Normalized Cross Correlation, NCC) finds the best matching location by calculating cross-correlation coefficients for each possible location in the template image and the original image. A correlation coefficient matching algorithm (Correlation Matching) finds the best matching location by calculating the correlation coefficient for each possible location in the template image and the original image. Similar to the normalized cross-correlation matching algorithm, the correlation coefficient matching algorithm can also deal with the problems of illumination change, scale change, rotation change and the like. The square error (Sum of Squared Differences, SSD) template matching algorithm is a pixel-based template matching method for finding the position of a target template in an image. It measures the similarity of the target template by calculating the sum of squares of the differences between its pixels at corresponding positions in the image to be matched.
Optionally, performing differential processing on the slope image to obtain a slope feature image, including: preprocessing a side slope image to obtain a side slope image to be detected; and carrying out differential processing on the slope image to be detected by adopting an inter-frame differential algorithm to obtain a slope characteristic image.
Optionally, preprocessing the side slope image to obtain a side slope image to be detected, including: denoising the slope image by adopting Gaussian filtering and mean square filtering to obtain a denoised image; image enhancement is carried out on the reflection component of the denoising image by adopting an image enhancement method based on guide filtering, so as to obtain an enhanced image; and carrying out edge refinement on the enhanced image by adopting a non-maximum value inhibition and self-adaptive threshold processing method to obtain the slope image to be detected.
In some embodiments, the inspection images and the internet are used to collect the images to form a dataset, while using manual means to simulate disease; acquiring a region of interest in the inspection image by using a template matching method based on an image pyramid; denoising the image in the data set by adopting Gaussian filtering and mean square filtering; then, a Retinex method based on guided filtering is adopted to obtain the reflection component of the image for image enhancement, and the details and colors of the image are restored; and further refining the edges by adopting a non-maximum suppression and self-adaptive threshold processing method of the traditional Canny algorithm.
In some embodiments, canny edge detection has the following steps:
(11) Denoising: the image is smoothed using a gaussian filter to remove noise. A gaussian filter is a linear filter that can smooth an image and reduce noise.
Calculating an image gradient: on the smoothed image, the gradient of the image is calculated using the Sobel operator. The gradient may find sharp edges in the image.
(12) Non-maximum suppression: on the gradient image, non-maximum suppression is performed to eliminate unnecessary response by edge detection. This process may allow the edges to be more refined.
(13) Double threshold detection: dual threshold detection is used to determine which edges are true edges and which are noise. The dual threshold detection divides the gradient image into three parts, strong edge, weak edge and noise. If the gradient value of the pixel point is greater than the high threshold, it is considered a strong edge. If the gradient value of the pixel is between the high and low threshold, it is considered a weak edge. If the gradient value of the pixel point is smaller than the low threshold value, it is regarded as noise. In general, the gap between the high threshold and the low threshold is referred to as a threshold difference.
(14) Edge connection: the final step is edge bonding, which is used to bond strong and weak edges to form a complete edge line. The method of connection is typically by tracking the path of the strong edge, linking adjacent weak edges to the strong edge.
Optionally, performing differential processing on the slope image to be detected by adopting an inter-frame differential algorithm, and obtaining the slope characteristic image includes: and respectively carrying out differential processing, double-channel conversion processing, self-adaptive threshold processing and morphological correction processing on the side slope image to be detected.
In some embodiments, an inter-frame differential algorithm is adopted to conduct differential processing on the side slope image to be detected, and a change area is obtained; the image quality is improved by adopting the double-channel conversion processing, the interference is reduced by adopting the self-adaptive threshold processing, and the change characteristic is processed by adopting the morphological correction processing, so that the final slope characteristic image is obtained.
In some embodiments, the steps of the inter-frame difference algorithm are as follows:
(21) The video sequence is read and each frame is converted into a grey scale image.
(22) For each pixel, a difference in pixel value is calculated between adjacent frames, resulting in a differential image. The difference for each pixel can be calculated using equation (4-1):
diff(x,y,t)＝abs(l(x,y,t)-l(x,y,t-1))
where diff (x, y, t) represents the difference in pixel value between the t-th frame and the t-1 th frame at the coordinates (x, y), l (x, y, t) represents the pixel value at the coordinates (x, y) in the t-th frame, and abs is an absolute value function.
(23) And carrying out binarization processing on the differential image, removing unnecessary noise and detail information, and converting the difference of pixel values into binary 0 or 1.
(24) And (3) performing motion detection on the binarized differential image, and detecting and tracking a moving object by using methods such as connected domain analysis, edge detection, template matching and the like.
The inter-frame difference algorithm has the advantages of simplicity and easiness in implementation, does not need a complex algorithm or model, and can perform motion detection and tracking in real time. In addition, the inter-frame difference algorithm has good detection effect on some simple scenes, such as static background and scenes of moving objects.
Optionally, the slope disease detection is performed according to the slope characteristic image, including: and inputting the slope characteristic image into a preset slope disease detection model to obtain a slope disease detection result.
Optionally, deep learning is performed on the convolutional neural network, training parameters including Epoch (round), batch size (size of data set), learning rate, dropout (regularization) and resolution factor are set, and the training parameters are continuously adjusted in the training process until an optimal slope disease detection model is obtained.
In some embodiments, after processing through image processing, template matching and edge detection, an inter-frame difference algorithm is used for surface cracks and drain plugs, a Lucas-Kanade optical flow detection algorithm is used for macroscopic deformation, and finally classification analysis is performed on diseases through ConvNext classification algorithm in deep learning.
Referring to fig. 8, a slope disease detection device based on unmanned aerial vehicle collection image includes: the unmanned aerial vehicle acquisition image acquisition module 101 is configured to acquire images of the slope area through an unmanned aerial vehicle, so as to acquire unmanned aerial vehicle acquisition images; the side slope image recognition module 102 is configured to perform template matching on the collected images of the unmanned aerial vehicle and recognize side slope images; the slope characteristic image acquisition module 103 is configured to perform differential processing on the slope image to obtain a slope characteristic image; a slope disease detection module 104 configured to perform slope disease detection according to the slope feature image; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
In some embodiments, a slope disease detection system based on unmanned aerial vehicle captured images, comprising: unmanned aerial vehicle, unmanned aerial vehicle embedded equipment and cloud server of carrying surveillance camera head: the unmanned aerial vehicle acquires images of the side slope area through the camera, acquires the images acquired by the unmanned aerial vehicle and transmits the images to the unmanned aerial vehicle embedded equipment; the unmanned aerial vehicle embedded device processes the image acquired by the unmanned aerial vehicle to obtain a side slope image, detects side slope diseases according to the side slope image, and sends the side slope image and the side slope disease detection result to the remote server.
Referring to fig. 7, this embodiment provides a slope disease detection method based on unmanned aerial vehicle collected images, including:
s01, acquiring an image of a slope area through an unmanned aerial vehicle to obtain an unmanned aerial vehicle acquired image;
step S02, performing image enhancement processing on an image acquired by the unmanned aerial vehicle to obtain an image to be detected;
s03, inputting an image to be detected into a preset slope image recognition model to obtain a slope image;
and S04, inputting the slope image into a preset slope disease detection model to obtain a slope disease detection result.
According to the scheme, the unmanned aerial vehicle is used for rapidly acquiring the image of the slope area and nearby processing the acquired data to obtain the slope image, and manual inspection is not needed. Compared with the mode of manual identification and detection after the unmanned aerial vehicle collects the pictures, the slope disease detection mode is used for detecting the slope diseases on the slope images, the response time of detection results is greatly shortened, and the integration and the real-time performance of data collection, slope image identification and slope disease detection are realized.
Optionally, performing image enhancement processing on the image acquired by the unmanned aerial vehicle to obtain an image to be detected, including: and performing perspective transformation processing on the unmanned aerial vehicle acquired image by adopting a perspective transformation matrix to obtain an image to be detected. The perspective transformation is non-linear transformation, the unmanned aerial vehicle collected image is mapped from one perspective projection view angle to another view angle, so that the geometric shape of the image is changed, and the unmanned aerial vehicle collected image can be subjected to perspective transformation to correct images which are distorted due to the optical influence of shooting or a camera, so that more and more accurate images to be detected are obtained.
Optionally, the image to be detected is obtained by the following formula:
wherein X, Y and Z are homogeneous coordinates of pixel points in an image acquired by the unmanned aerial vehicle, X, Y and Z are coordinates of the transformed pixel points of the image to be detected, and (a, b, c, d, e, f, g, h and i) are perspective transformation matrixes.
Optionally, performing image enhancement processing on the image acquired by the unmanned aerial vehicle to obtain an image to be detected, including:
and carrying out linear smoothing filtering treatment on the image acquired by the unmanned aerial vehicle to obtain an image to be detected. And after weighting and averaging target pixels and surrounding pixels in the image by linear smoothing filter processing, the average value is filled back to the target pixels, so that denoising of the unmanned aerial vehicle acquired image is realized, and a more simplified image to be detected is obtained.
Optionally, the unmanned aerial vehicle acquired image is subjected to linear smoothing filter processing by the following formula:
g(i,j)＝∑ k,l f(i+k,j+l)h(k,l)
wherein g (i, j) is a filtered image to be detected, f (i, j) is an unmanned aerial vehicle collected image, h (k, l) is a field operator, and k and l represent the sizes of the computer cores.
Optionally, before inputting the image to be detected into the preset slope image recognition model, the method further includes: constructing a first data set, wherein the first data set comprises a plurality of image samples to be detected; dividing the first data set into a first training set and a first test set; setting training parameters and training strategies, wherein the training parameters comprise training round numbers, initial weight parameters, learning rate and optimizers; the training strategy is to update the weight parameter once every training round, and update the training parameter every preset round number; based on training parameters and training strategies, training an initial model by adopting a first training set, performing accuracy test on the training model by adopting a first testing set, and determining the training model as a slope image recognition model under the condition that the accuracy of the training model meets the preset requirement.
Optionally, if the accuracy of the training model is greater than a preset threshold, determining the training model as a slope image recognition model.
In some embodiments, acquiring the slope image recognition model includes the steps of:
s301: constructing a first data set, wherein the first data set comprises a plurality of image samples to be detected;
s302: dividing the first data set into a first training set and a first test set;
s303: setting training parameters, wherein the training parameters comprise a training round number of 300, a learning rate of 0.05, and an optimizer is set as adam (adaptive momentum random optimization method);
s304: the training strategy is set as a linear prediction strategy, a loss function is calculated after each round of training, the weight parameters are updated to start the next round of training, and the training parameters are updated once every ten rounds of training.
S305: based on training parameters and training strategies, training an initial model by adopting a first training set, testing the precision of the trained model by utilizing a first testing set after each training round until the precision of the training model meets the requirement, and stopping training to obtain a slope image recognition model.
Optionally, the slope image recognition model includes: the convolution feature extraction module is used for carrying out feature processing on the image to be detected to obtain a plurality of feature information; the feature screening module is used for screening the plurality of feature information to obtain a plurality of screening information; the context feature fusion module is used for carrying out feature fusion on the plurality of screening information to obtain fusion features; and the encoding and decoding module is used for decoding the fusion characteristics and extracting the characteristics to obtain the slope image.
In some embodiments, as shown in fig. 2, the image to be detected is subjected to feature extraction by a convolution feature extraction module to obtain feature information such as image texture, color and edge information, and the convolution feature extraction module transmits the feature information to a feature screening module. The feature screening module screens the plurality of feature information to make the model pay more attention to the slope target area, filters out invalid information such as background, and the feature screening module transmits the screened screening information to the context feature fusion module. The context feature fusion module fuses the features of different scales according to the propagation path of the feature pyramid from top to bottom, enhances the expression capability of the feature map, and finally adopts the concept of dividing and treating the feature map, and distinguishes the receptive fields of the fused feature map according to the large, medium and small targets of the original image, so that the matching of the feature map is more refined. The context feature fusion module transmits the fusion features to the encoding and decoding module, the encoding and decoding module decodes the fusion features with different receptive fields output by the context feature fusion module, the features are further extracted by adopting a convolution and attention mechanism, finally, the detection areas and the types of the targets are respectively returned by adopting a decoupling head mode, and a side slope image is output.
Optionally, the context feature fusion module fuses the features sequentially according to a propagation path of the feature pyramid from top to bottom, and finally, the receptive fields of the fused features are distinguished according to the large, medium and small targets of the image to be detected and transmitted to the feature screening module.
Optionally, the codec module includes: the channel feature screening submodule is used for carrying out channel feature screening on the fusion features to obtain first screening information; the spatial feature screening sub-module is used for carrying out spatial feature screening on the first screening information to obtain second screening information; and the output sub-module is used for identifying the side slope image according to the second screening information.
Optionally, the channel feature screening submodule includes a first average pooling layer, a first maximum pooling layer, a full connection layer and a first activation function layer; the fusion features enter a first average pooling layer and a first maximum pooling layer to carry out average pooling and maximum pooling treatment, spatial compression features are output, the spatial compression features output channel feature information through a full-connection layer, and the channel feature information outputs first screening information through a first activation function layer.
Alternatively, as shown in connection with FIG. 3, the channel feature screening submodule is represented by the following formula:
M c (F)＝sigmoid(MLP(AvgPool(F))+MLP(MaxPool(F)))
where F is the fusion feature of the input, MLP is the fully connected layer, avgPool and MaxPool are global maximum pooling (GlobalMaxPool) and global average pooling (GlobalMaxPool), respectively, sigmoid is the activation function.
In some embodiments, in the channel feature screening submodule, the fusion features compress the spatial features through average pooling and maximum pooling respectively, and output the spatial compression features; then outputting the space compression characteristic, further refining channel characteristic information through the full connection layer, and outputting the channel characteristic information; and finally, outputting spatial compression characteristics to realize screening of characteristic information of different channels through sigmoid function mapping.
Optionally, the spatial feature screening submodule includes a second average pooling layer, a second maximum pooling layer, a convolution layer and a second activation function layer; the first screening information enters a second average pooling layer and a second maximum pooling layer to carry out average pooling and maximum pooling treatment, channel compression characteristics are output, the channel compression characteristics output spatial characteristic information through a convolution layer, and the spatial characteristic information passes through a second activation function layer to output second screening information.
Alternatively, as shown in connection with FIG. 4, the spatial feature screening sub-module is represented by the following formula:
M s (F`)＝Sigmoid(f 7x7 ([AvgPool(F)；MaxPool(F`)]))
wherein F' is the first screening information, F 7x7 Representing convolution operations, convolution kernel sizes are 7, avgpool and MaxPool are Global maximum pooling (Global MaxPool) and Global averaging (Global MaxPool), respectively, sigmoid is an activation function.
In some embodiments, in the spatial feature screening module, the first screening information compresses channel information of the first screening information to one dimension through average pooling and maximum pooling to obtain channel compression information; then, after carrying out convolution operation on the channel compressed information, outputting spatial characteristic information; the channel compression information is mapped through a sigmoid function, so that the characteristic information of the space is screened.
Optionally, the output submodule is used for carrying out vector connection on the second screening information to obtain a feature vector; obtaining Euclidean distance according to the feature vector and the feature vector sample in the sample library; and comparing the Euclidean distance with a preset threshold value, and identifying the slope image.
Optionally, the calculation formula of the euclidean distance is:
wherein x is i ,y i The values of the X, Y feature vectors at the i-th position are represented, and d (X, Y) represents the euclidean distance.
In some embodiments, the output sub-module compares the euclidean distance with a set threshold, and if the euclidean distance is greater than the set threshold, determines that the image to be detected is a side slope image, otherwise, it is not.
Optionally, the output submodule is used for carrying out vector connection on the second screening information, and screening out a target detection area by adopting non-maximum suppression based on preset confidence level, so as to obtain the side slope image.
Optionally, inputting the slope image into a preset slope disease detection model to obtain a slope disease detection result. Referring to fig. 5, the slope disease detection model includes a feature extraction backbone network and a feature encoding module; the feature extraction backbone network comprises a plurality of feature extraction layers, each feature extraction layer is provided with a feature screening layer, and feature images with obvious feature expression are obtained through layer-by-layer feature extraction and screening, the feature coding module comprises two layers of fully-connected networks, and the feature coding module outputs 128-dimensional feature vectors.
Optionally, the slope disease detection model is trained by: constructing a second data set, wherein the second data set comprises a plurality of slope image samples; dividing the second data set into a second training set and a second test set; and training the initial model by adopting a second training set, testing the accuracy of the training model by adopting a second testing set, and stopping training under the condition that the accuracy of the training model meets the preset requirement to obtain the side slope disease detection model.
Referring to fig. 6, this embodiment provides a system for detecting a disease of a side slope based on an image collected by an unmanned aerial vehicle, including: unmanned aerial vehicle 1, unmanned aerial vehicle embedded equipment 3 and high in the clouds server 4 of carrying camera 2.
Optionally, the unmanned aerial vehicle embedded device is deployed with a slope image recognition model and a slope disease detection model.
According to the side slope disease detection system based on the unmanned aerial vehicle collected image, the unmanned aerial vehicle camera is used for collecting images of a side slope area, so that the unmanned aerial vehicle collected image is obtained and transmitted to unmanned aerial vehicle embedded equipment; the unmanned aerial vehicle embedded device processes the image acquired by the unmanned aerial vehicle to obtain a side slope image, detects side slope diseases according to the side slope image, and sends the side slope image and the side slope disease detection result to the remote server.
Optionally, the cloud server is further used for displaying real-time monitoring slope images, alarm information and the like on the web side, and meanwhile, the server also stores slope images so as to update a slope image recognition model and a slope disease detection model.
Optionally, the unmanned aerial vehicle embedded device is deployed with a slope image recognition model, and the remote server is deployed with a slope disease detection model.
In some embodiments, an unmanned aerial vehicle embedded device (e.g., jetson Xavier NX) reads unmanned aerial vehicle acquisition image information in a camera, and performs an image enhancement operation on the input image on the embedded device; and then inputting a slope image recognition model which is deployed on the unmanned aerial vehicle embedded equipment to obtain a slope image, inputting the slope image into a cloud server, and inputting the slope image into a preset slope disease detection model by the cloud server to obtain a slope disease detection result.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application, and are intended to be included within the scope of the appended claims and description.
Claims (9)
1. The slope disease detection method based on the unmanned aerial vehicle collected images is characterized by comprising the following steps of:
image acquisition is carried out on the slope area through the unmanned aerial vehicle, and an unmanned aerial vehicle acquired image is obtained;
performing template matching on the image acquired by the unmanned aerial vehicle, and identifying a slope image;
carrying out differential processing on the slope image to obtain a slope characteristic image;
slope disease detection is carried out according to the slope characteristic images; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
2. The method of claim 1, further comprising, prior to template matching the unmanned aerial vehicle captured image:
and carrying out image enhancement processing on the image acquired by the unmanned aerial vehicle.
3. The method of claim 1, wherein template matching the unmanned aerial vehicle captured image, identifying the side slope image comprises:
constructing an image pyramid according to the image acquired by the unmanned aerial vehicle;
selecting a target template, and carrying out a template matching algorithm on each layer of image pyramid based on the target template to obtain the optimal matching position of each layer of image pyramid;
obtaining an optimal matching result according to the optimal matching position of each layer of image pyramid;
and judging whether the slope image is a slope image or not according to the best matching result.
4. A method according to claim 3, wherein the template matching algorithm comprises a brute force matching algorithm, a normalized cross-correlation matching algorithm, a correlation coefficient matching algorithm or a squared difference matching algorithm.
5. The method of claim 1, wherein the differentiating the slope image to obtain the slope feature image comprises:
preprocessing a side slope image to obtain a side slope image to be detected;
and carrying out differential processing on the slope image to be detected by adopting an inter-frame differential algorithm to obtain a slope characteristic image.
6. The method of claim 5, wherein preprocessing the side slope image to obtain the side slope image to be detected comprises:
denoising the slope image by adopting Gaussian filtering and mean square filtering to obtain a denoised image;
image enhancement is carried out on the reflection component of the denoising image by adopting an image enhancement method based on guide filtering, so as to obtain an enhanced image;
and carrying out edge refinement on the enhanced image by adopting a non-maximum value inhibition and self-adaptive threshold processing method to obtain the slope image to be detected.
7. The method of claim 1, wherein the slope disease detection based on the slope feature image comprises:
and inputting the slope characteristic image into a preset slope disease detection model to obtain a slope disease detection result.
8. Side slope disease detection device based on unmanned aerial vehicle gathers image, its characterized in that includes:
the unmanned aerial vehicle acquisition image acquisition module is configured to acquire images of the slope area through the unmanned aerial vehicle, so as to acquire unmanned aerial vehicle acquisition images;
the side slope image recognition module is configured to perform template matching on the collected images of the unmanned aerial vehicle and recognize side slope images;
the slope characteristic image acquisition module is configured to perform differential processing on the slope image to acquire a slope characteristic image;
the slope disease detection module is configured to detect slope disease according to the slope characteristic image; the slope disease detection comprises slope cracks, local displacement of the slope and blockage of the drainage ditch.
9. A side slope disease detection system based on unmanned aerial vehicle collection image, comprising: unmanned aerial vehicle, unmanned aerial vehicle embedded equipment and cloud server of carrying surveillance camera head: the unmanned aerial vehicle acquires images of the side slope area through the camera, acquires the images acquired by the unmanned aerial vehicle and transmits the images to the unmanned aerial vehicle embedded equipment; the unmanned aerial vehicle embedded device processes the image acquired by the unmanned aerial vehicle to obtain a side slope image, detects side slope diseases according to the side slope image, and sends the side slope image and the side slope disease detection result to the remote server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310960856.2A CN116994160A (en) | 2023-07-28 | 2023-07-28 | Slope disease detection method, device and system based on unmanned aerial vehicle acquired image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310960856.2A CN116994160A (en) | 2023-07-28 | 2023-07-28 | Slope disease detection method, device and system based on unmanned aerial vehicle acquired image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116994160A true CN116994160A (en) | 2023-11-03 |
Family
ID=88520963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310960856.2A Pending CN116994160A (en) | 2023-07-28 | 2023-07-28 | Slope disease detection method, device and system based on unmanned aerial vehicle acquired image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116994160A (en) |
-
2023
- 2023-07-28 CN CN202310960856.2A patent/CN116994160A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhao et al. | Multi-scale optimal fusion model for single image dehazing | |
| CN108615226B (en) | Image defogging method based on generation type countermeasure network | |
| CN111553265B (en) | Method and system for detecting internal defects of drainage pipeline | |
| WO2018122809A1 (en) | Static infrared thermal image processing-based underground pipe leakage detection method | |
| CN111738342B (en) | Pantograph foreign matter detection method, storage medium and computer equipment | |
| CN110415208A (en) | A kind of adaptive targets detection method and its device, equipment, storage medium | |
| CN110569755B (en) | Intelligent accumulated water detection method based on video | |
| US20220128358A1 (en) | Smart Sensor Based System and Method for Automatic Measurement of Water Level and Water Flow Velocity and Prediction | |
| CN111462027B (en) | Multi-focus image fusion method based on multi-scale gradient and matting | |
| CN113313107B (en) | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge | |
| Daniel et al. | Automatic road distress detection and analysis | |
| CN115830004A (en) | Surface defect detection method, device, computer equipment and storage medium | |
| CN117094914A (en) | Smart city road monitoring system based on computer vision | |
| CN114359149A (en) | Dam bank dangerous case video detection method and system based on real-time image edge enhancement | |
| CN117079117B (en) | Underwater image processing and target identification method and device, storage medium and electronic equipment | |
| CN116228712A (en) | Multi-scale slope disaster monitoring method, system and device | |
| CN116152758A (en) | Intelligent real-time accident detection and vehicle tracking method | |
| Balcilar et al. | Extracting vehicle density from background estimation using Kalman filter | |
| CN116994160A (en) | Slope disease detection method, device and system based on unmanned aerial vehicle acquired image | |
| CN112183310B (en) | Method and system for filtering redundant monitoring pictures and screening invalid monitoring pictures | |
| CN114694090A (en) | Campus abnormal behavior detection method based on improved PBAS algorithm and YOLOv5 | |
| CN108171168B (en) | Intelligent image detection method and device for small and weak target change | |
| Chaiyawatana et al. | Robust object detection on video surveillance | |
| CN116311212B (en) | Ship number identification method and device based on high-speed camera and in motion state | |
| Sumi et al. | Motion deblurring for pedestrian crossing detection in advanced driver assistance system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |