CN116503760A - Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation - Google Patents

Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation Download PDF

Info

Publication number
CN116503760A
CN116503760A CN202310471491.7A CN202310471491A CN116503760A CN 116503760 A CN116503760 A CN 116503760A CN 202310471491 A CN202310471491 A CN 202310471491A CN 116503760 A CN116503760 A CN 116503760A
Authority
CN
China
Prior art keywords
edge
shape
edge feature
unmanned aerial
aerial vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310471491.7A
Other languages
Chinese (zh)
Inventor
张晖
石亦巍
赵海涛
朱洪波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN202310471491.7A priority Critical patent/CN116503760A/en
Publication of CN116503760A publication Critical patent/CN116503760A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention discloses an unmanned aerial vehicle cruise detection method based on self-adaptive edge feature semantic segmentation, which is based on a predefined multi-scale multi-shape edge detection operator group and obtains a multi-shape edge feature map by utilizing a multi-shape edge feature fusion method based on edge probability; obtaining a multi-scale multi-shape edge feature map by using a multi-scale edge feature weighting method based on target matching degree; building an improved deep LabV3+ semantic segmentation network, using a receptive field fusion cavity space pyramid pooling module in an encoder, supplementing a multi-scale multi-shape edge feature image group into a network decoder part, and training the improved network; and finally, performing segmentation detection on the target in the remote sensing image obtained by unmanned aerial vehicle inspection by using the trained network. According to the invention, through training and extracting the characteristic representation of the generalized inspection target on the unmanned aerial vehicle inspection data set, the distinguishable degree among different target characteristics is highlighted, the detection accuracy is improved, and the manual inspection work is reduced.

Description

Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation
Technical Field
The invention relates to an unmanned aerial vehicle cruise detection method based on self-adaptive edge feature semantic segmentation, and belongs to the fields of semantic segmentation and machine vision.
Background
In recent years, the development of the internet of things is rapid, and as an important component of a novel digital infrastructure, the digitization and the intellectualization of the energized industry are being accelerated, and the technical industry system is deeply changed. The unmanned aerial vehicle inspection system has the advantages that requirements of various industries on unmanned aerial vehicles and automation are further improved, and the unmanned aerial vehicle inspection plays an important role. The traditional inspection is finished by manpower, a large amount of manpower, material resources and financial resources are needed, the problems of detection efficiency and safety exist, and the unmanned aerial vehicle inspection is enabled to solve the problems due to the advantages of simplicity in operation, small site limitation, high operation efficiency and the like. Unlike traditional inspection, unmanned aerial vehicle inspection generally has characteristics of large overlook, long distance, high dynamic, which makes the inspection image obtained by the unmanned aerial vehicle inspection different from the common inspection image. The remote sensing image obtained by unmanned aerial vehicle inspection generally has the characteristics of complex background, large object scale difference, large resolution and the like, and meanwhile, the existing detection means are poor in performance under the scene due to the problems of dynamic change of inspection target illumination, multi-target edge semantic ambiguity and the like.
In summary, how to accurately identify multiple targets in an unmanned aerial vehicle inspection image in the prior art is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the unmanned aerial vehicle cruising detection method based on the self-adaptive edge feature semantic segmentation solves the problems of inaccurate edge detection and low target segmentation accuracy in unmanned aerial vehicle patrol images with large target scale difference in the prior art.
The invention adopts the following technical scheme for solving the technical problems:
an unmanned aerial vehicle cruise detection method based on self-adaptive edge feature semantic segmentation comprises the following steps:
step 1, defining a multi-scale multi-shape edge detection operator group;
step 2, utilizing a multi-shape edge feature fusion method based on edge probability, and utilizing the multi-scale multi-shape edge detection operator group in the step 1 to obtain a multi-shape edge feature image group;
step 3, weighting the multi-scale multi-shape edge feature image obtained in the step 2 according to the target matching degree by a multi-scale edge feature weighting method based on the target matching degree to obtain a multi-scale multi-shape edge feature image group;
step 4, setting up a semantic segmentation network and training the semantic segmentation network;
and 5, detecting various targets in the unmanned aerial vehicle inspection image by using the trained semantic segmentation network.
Further, the set of scale multi-shape edge detection operators described in step 1 is defined as follows: the operator group is divided into n classes according to the extracted edge characteristic shape, is divided into m classes according to the operator scale, and total n multiplied by m edge detection operators.
Further, the edge probability-based multi-shape edge feature fusion method described in step 2 includes the following steps:
step 2.1, converting an image to be detected from an RGB image into a gray image, and obtaining n multiplied by m edge feature images according to the multi-scale multi-shape edge detection operator group proposed in the step 1;
step 2.2, defining the shape edge intensity as the arithmetic average value of the sum of gray values of adjacent pixels in the edge shape direction, and the relative edge gradient as the arithmetic average value of the sum of gradient values in the pixel edge shape direction;
step 2.3, defining the pixel point with gray value not being 0 in the edge feature map as the pixel point of the edge to be determined, and calculating the average gray value, the average shape edge strength and the average opposite edge gradient of the pixel point of the edge to be determined;
step 2.4, calculating the probability that the current pixel point is an edge pixel point according to the gray value, the shape edge strength and the relative edge gradient of the current pixel point;
step 2.5, binarizing the edge feature map according to gamma (p), and obtaining n binarized shape edge feature maps corresponding to each operator size;
step 2.6, performing OR operation on n binarized shape edge feature graphs corresponding to each operator size, and fusing the n binarized shape edge feature graphs into a polygonal edge shape feature graph E S A multi-shape edge feature map set including m multi-shape edge feature maps is obtained.
Further, the binarization operation specifically includes: regarding the pixel points with gamma (p) higher than the set threshold value as edge pixel points, and setting the gray value as 255; the pixel below the threshold is considered as a non-edge pixel, and the gray value is set to 0.
Further, the shape edge intensity of the pixel point pOpposite edge gradient->Wherein Gray (p-u) represents the Gray value of the u-th pixel in the negative direction of the edge shape of the pixel p, gray (p+u) represents the Gray value of the u-th pixel in the positive direction of the edge shape of the pixel p, and Gray (p) represents the Gray value of the pixel p.
Further, the probability that the current pixel point p is an edge pixel pointGray(p)、/>Respectively representing the gray value, shape edge intensity and relative edge gradient of the current pixel point +.>Representing the average gray value, average shape edge intensity and average relative edge gradient of the pixel points of the undetermined edge respectively.
Further, the multi-scale edge feature weighting method based on the target matching degree in the step 3 comprises the following steps:
step 3.1, defining a height influence factor, wherein the specific calculation method comprises the following steps:
wherein [ Alt ] min ,Alt max ]Represents the unmanned aerial vehicle cruising altitude interval, alt cur Representing the altitude at which the inspection image of the drone was acquired,represents the cruising average height, k of the unmanned aerial vehicle i 、k j Respectively representing the operator scales corresponding to the ith and jth multi-shape edge feature images in the multi-shape edge feature image group, < ->Representing a plurality of shapesThe height influence factor corresponding to the ith multi-shape edge feature map in the edge feature map group;
defining an edge quantity factorThe specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,represents an intermediate variable,/->Representing the number of edge points in the ith and jth polygonal edge shape feature images in the multi-shape edge feature image group;
and 3.2, calculating the target matching degree according to the height influence factor and the edge quantity factor obtained in the step 3.1, wherein the specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the target matching degree corresponding to the ith polygonal edge shape feature map in the multi-shape edge feature map group;
and 3.3, weighting the polygonal edge shape feature image group according to the target matching degree to obtain a multi-scale multi-shape edge feature image group, wherein the specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,weighting the i-th multi-scale multi-shape edge feature map in the multi-shape edge feature map set, ++>Is the ith polygon edge shape characteristic diagram in the multi-shape edge characteristic diagram group.
Further, an improved deep LabV3+ semantic segmentation network is built in the step 4, and the specific steps are as follows:
step 4.1, using a receptive field fusion cavity space pyramid pooling module at an encoder part of the deep LabV3+ semantic segmentation network to carry out weighted fusion on branch self-adaption with different receptive fields, wherein the construction process of the receptive field fusion cavity space pyramid pooling module is as follows:
step 4.1.1 for convolutional layer P 1 And 3 hole convolution layers P 2 、P 3 、P 4 Splicing operation is carried out in the channel direction to obtain fusion characteristicsWherein (1)>Representing three-dimensional features of the feature map, H representing feature map height, W representing feature map width, and C representing feature map depth;
step 4.1.2, pooling the fusion feature P in the height direction and the width direction to obtainAnd->
Step 4.1.3 for Z h And Z w Performing channel splicing operation, adjusting channel number by using 1×1 convolution operation, activating by using a ReLU function to obtainWherein r is a scaling factor;
step 4.1.4 splitting f into two independent featuresAnd->Then the 1X 1 convolution operation is used for carrying out the dimension lifting operation, and the Sigmoid function is used for activating, so that +.>And->
Step 4.1.5, using g h And g w Calculating a weighted output P 'and pooling the weighted output P' with the global average pooling layer output P 5 Performing channel splicing operation to obtain the integral output P' of the receptive field fusion cavity space pyramid pooling module;
and 4.2, in a decoder part of the deep LabV3+ semantic segmentation network, performing four-time downsampling on the multi-scale multi-shape edge feature image group, and then splicing the multi-scale multi-shape edge feature image group with the advanced semantic features obtained by the encoder part in the channel direction.
The invention also provides an unmanned aerial vehicle cruise detection system based on the self-adaptive edge feature semantic segmentation, which is used for carrying out unmanned aerial vehicle cruise detection based on the method, and specifically comprises the following steps:
the multi-shape edge feature fusion module is used for obtaining a multi-shape edge feature image group according to a predefined multi-scale multi-shape edge detection operator group;
the multi-scale edge feature weighting module is used for weighting according to the target matching degree to obtain a multi-scale multi-shape edge feature image group;
the semantic segmentation network module is used for building a semantic segmentation network and training.
The present invention also provides a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the method as described above.
The invention also provides an unmanned aerial vehicle cruise detection device based on adaptive edge feature semantic segmentation, comprising one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method as described above.
Compared with the prior art, the technical scheme provided by the invention has the following technical effects:
1. the invention provides a multi-shape edge feature fusion method based on edge probability and a multi-scale edge feature weighting method based on target matching degree, which can more accurately extract the edge of an object;
2. the invention provides a receptive field fusion cavity space pyramid pooling module which is suitable for unmanned aerial vehicle remote sensing images with large scale difference of target objects, and improves the accuracy of image semantic segmentation;
3. the method can accurately divide and identify the target object in the video and picture information shot by the unmanned aerial vehicle in the unmanned aerial vehicle inspection, and provides accurate information for inspection decision.
Drawings
FIG. 1 is a flow chart of a multi-objective semantic segmentation detection algorithm;
FIG. 2 is a diagram of a receptive field fusion cavity space pyramid pooling module;
fig. 3 is an overall structure diagram of the modified deelabv3+ network.
Detailed Description
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
The invention discloses an unmanned aerial vehicle cruise detection method based on self-adaptive edge feature semantic segmentation, which is shown in figure 1 and comprises the following steps:
step 1, defining a multi-scale multi-shape edge detection operator group
The extracted edge feature shapes can be classified into n classes, such as vertical/horizontal/45 ° left-inclined/45 ° right-inclined, etc., and the operator template dimensions can be classified into m classes, such as 3×3, 5×5, 7×7, etc., for a total of n×m edge detection operators.
Step 2, using the multi-scale multi-shape edge detection operator group in step 1 by utilizing a multi-shape edge feature fusion method based on edge probability, and obtaining m multi-shape edge feature graphs according to the operator scale, wherein the steps are as follows:
(1) Converting an image to be detected into a gray level image from an RGB image, obtaining n edge shapes and n m operator scales to extract n times m total edge feature images according to the multi-scale multi-shape edge detection operator set proposed in the step 1, wherein the operator of each scale can obtain a group of n total edge feature images reflecting different edge shapes, and the group of n total edge feature images is marked as E v ,v∈[1,n]。
(2) The scale of the edge detection operator is marked as k, the Gray value of the pixel point p is marked as Gray (p), wherein p represents the position of the pixel point, and the shape edge intensity of the pixel point p is defined according to the size of the scale of the operator and the direction of the edge shapeThe specific calculation method is as follows:
wherein Gray (p-u) represents the Gray value of the u-th pixel in the negative direction of the edge shape, and Gray (p+u) represents the Gray value of the u-th pixel in the positive direction of the edge shape.
Defining pixel points according to the size of the operator and the direction of the edge shapep opposite edge gradient The actual sum of gradient values in positive and negative directions of the edge shape of the current pixel point p is calculated by the following method:
(3) The pixel point with gray value not being 0 in the edge map is called as the pixel point of the edge to be determined, the number of the pixel points is recorded as Num, and the average gray value of the pixel point of the edge to be determined is further calculatedAverage shape edge intensity>Average relative edge gradient +.>The specific calculation method is as follows:
then according to the Gray value Gray (p) of the current pixel point p and the shape edge intensityOpposite edge gradientThe probability gamma (p) that the current pixel point is an edge pixel point is calculated, and the specific calculation method is as follows:
then, binarizing operation is carried out on the edge feature map according to the probability gamma (p), the threshold value is set to be 0.5, the pixel points higher than the threshold value are regarded as edge pixel points, the gray level value is set to be 255, the pixel points lower than the threshold value are regarded as non-edge pixel points, the gray level value is set to be 0, and the specific calculation method is as follows:
through the operation, the edge detection operator of each scale can obtain n binarized shape edge feature graphs, which are marked as E i ',i∈[1,n]。
(4) For binarized shape edge feature map E i ',i∈[1,n]The pixel-by-pixel operation is carried out, n binarized shape edge feature images are fused into a polygonal edge shape feature image E S The specific calculation method is as follows:
E S ＝E 1 '⊙E 2 '......⊙E' n
wherein, as indicated by ". Sur represents a pixel-by-pixel or operation.
Through the operation, the edge detection operator of each scale can obtain a multi-shape edge characteristic diagram E S Finally, m multi-shape edge feature images are obtained.
Step 3, weighting the m multi-scale edge feature images obtained in the step 2 according to the target matching degree by using a multi-scale edge feature weighting method based on the target matching degree to obtain a multi-scale edge feature image group, wherein the multi-scale edge feature image group is specifically as follows:
(1) Obtaining a multi-shape edge feature image group consisting of m Zhang Duoxing-shape edge feature images according to the step 2, which is marked as
(2) Along with the increase of the cruising height of the unmanned aerial vehicle, the size of an object in a remote sensing image of the unmanned aerial vehicle can be correspondingly reduced, and the unmanned aerial vehicle is detected by a small-scale edge detection operatorThe greater the weight assigned to the obtained feature map, the smaller the weight assigned to the feature map obtained by the large-scale edge detection operator, and the unmanned aerial vehicle cruising altitude interval is denoted as [ Alt ] min ,Alt max ]For measuring influence of unmanned aerial vehicle cruising altitude on polygonal edge shape characteristic diagram weights obtained by different scale operators, defining altitude influence factorsThe specific calculation method is as follows:
wherein Alt is cur Represents the height of the unmanned aerial vehicle when the remote sensing image is acquired, alt represents the cruising average height of the unmanned aerial vehicle, and k i Representing the operator scale corresponding to the ith multi-shape edge feature map.
The number of edge points in each polygonal edge shape feature graph is recorded asThe more the number of edge points is, the more suitable the operator of the scale is for the image, and an edge quantity factor is defined for measuring the influence of the number of edges on the weight of the polygonal edge shape feature map obtained by the operators of different scales>The specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing an intermediate variable.
(3) According to the height influencing factorAnd edge quantity factor->Calculate the target matching degree +.>The specific calculation method is as follows:
finally according to the matching degree of the targetFor the polygon edge shape characteristic diagram group E S Weighting to obtain multi-scale multi-shape edge feature map group +.>The specific calculation method is as follows:
step 4, an improved deep LabV3+ semantic segmentation network is built, and the method is as follows in detail as shown in fig. 3:
(1) Constructing a receptive field fusion cavity space pyramid pooling module, as shown in fig. 2, firstly, for a common convolution layer P 1 And 3Individual hole convolution layer P 2 、P 3 、P 4 Splicing operation is carried out in the channel direction to obtain fusion characteristicsWherein (1)>Representing three-dimensional features of the feature map, H representing feature map height, W representing feature map width, and C representing feature map depth.
Then the fusion features P are pooled in the height direction and the width direction to obtainAndthe calculation process is as follows:
then to Z h And Z w Performing channel splicing operation, adjusting channel number by using 1×1 convolution operation, and activating by using a ReLU function to obtainThe calculation process is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,representative channel splicing operationActing as kernel 1×1 (. Cndot.) represents a 1X 1 convolution, r being the scaling factor.
Splitting f to obtain two independent characteristicsAnd->Then the 1X 1 convolution operation is used for carrying out the dimension lifting operation, and the Sigmoid function is used for activating, so that +.>And->The calculation process is as follows:
g h ＝Sigmoid(kernel 1×1 (f h ))
g w ＝Sigmoid(kernel 1×1 (f w ))。
finally, use g h And g w Calculating a weighted output P 'and combining the weighted output P' with a pooled layer output P 5 And (3) performing channel splicing operation to obtain the integral output P' of the receptive field fusion cavity space pyramid pooling module, wherein the calculation process comprises the following steps:
(2) And 3, in the encoder part, replacing the original cavity space pyramid pooling module by using a cavity space pyramid pooling module fused by a receptive field, supplementing the multi-scale multi-shape edge feature image group obtained in the step 3 into a network in the decoder part, splicing the multi-scale multi-shape edge feature image group with the advanced semantic features obtained in the encoder part in the channel direction after four-time downsampling, and training a network model.
The invention discloses an unmanned aerial vehicle cruise detection method based on self-adaptive edge feature semantic segmentation. Firstly, defining a multi-scale multi-shape edge detection operator group; secondly, extracting an edge feature map by using a multi-scale multi-shape edge detection operator group, and obtaining a multi-shape edge feature map by using a multi-shape edge feature fusion method based on edge probability; further, weighting the multi-scale multi-shape edge feature images according to the target matching degree by using a multi-scale edge feature weighting method based on the target matching degree to obtain a multi-scale multi-shape edge feature image group; then, an improved deep LabV3+ semantic segmentation network is built, a receptive field fusion cavity space pyramid pooling module is used in an encoder, a multi-scale multi-shape edge feature image group is supplemented into a network decoder part, and the improved deep LabV3+ semantic segmentation network is trained; and finally, utilizing the trained deep LabV3+ semantic segmentation network to segment and detect the target in the remote sensing image obtained by unmanned aerial vehicle inspection. According to the invention, through training and extracting the characteristic representation of the generalized inspection target on the unmanned aerial vehicle inspection data set, the distinguishable degree among different target characteristics is highlighted, the detection accuracy is improved, and the manual inspection work is reduced.
The invention also provides an unmanned aerial vehicle cruise detection system based on the self-adaptive edge feature semantic segmentation, which specifically comprises the following steps:
the multi-shape edge feature fusion module is used for obtaining a multi-shape edge feature image group according to a predefined multi-scale multi-shape edge detection operator group;
the multi-scale edge feature weighting module is used for weighting according to the target matching degree to obtain a multi-scale multi-shape edge feature image group;
the semantic segmentation network module is used for building a semantic segmentation network and training.
The technical scheme of the unmanned aerial vehicle cruise detection system is similar to that of the method, and is not repeated here.
Based on the same technical scheme, the invention also discloses a computer readable storage medium storing one or more programs, wherein the one or more programs comprise instructions, and the instructions, when executed by a computing device, cause the computing device to execute the unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation.
Based on the same technical scheme, the invention also discloses a computing device, which comprises one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing the unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above embodiments are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereto, and any modification made on the basis of the technical scheme according to the technical idea of the present invention falls within the protection scope of the present invention.

Claims (10)

1. The unmanned aerial vehicle cruise detection method based on the self-adaptive edge feature semantic segmentation is characterized by comprising the following steps of:
step 1, defining a multi-scale multi-shape edge detection operator group;
step 2, utilizing a multi-shape edge feature fusion method based on edge probability, and utilizing the multi-scale multi-shape edge detection operator group in the step 1 to obtain a multi-shape edge feature image group;
step 3, weighting the multi-scale multi-shape edge feature image obtained in the step 2 according to the target matching degree by a multi-scale edge feature weighting method based on the target matching degree to obtain a multi-scale multi-shape edge feature image group;
step 4, setting up a semantic segmentation network and training the semantic segmentation network;
and 5, detecting various targets in the unmanned aerial vehicle inspection image by using the trained semantic segmentation network.
2. The unmanned aerial vehicle cruise detection method based on adaptive edge feature semantic segmentation according to claim 1, wherein the set of scale multi-shape edge detection operators in step 1 is defined as follows: the operator group is divided into n classes according to the extracted edge characteristic shape, is divided into m classes according to the operator scale, and total n multiplied by m edge detection operators.
3. The unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation according to claim 2, wherein the edge probability-based multi-shape edge feature fusion method in step 2 comprises the following steps:
step 2.1, converting an image to be detected from an RGB image into a gray image, and obtaining n multiplied by m edge feature images according to the multi-scale multi-shape edge detection operator group proposed in the step 1;
step 2.2, defining the shape edge intensity as the arithmetic average value of the sum of gray values of adjacent pixels in the edge shape direction, and the relative edge gradient as the arithmetic average value of the sum of gradient values in the pixel edge shape direction;
step 2.3, defining the pixel point with gray value not being 0 in the edge feature map as the pixel point of the edge to be determined, and calculating the average gray value, the average shape edge strength and the average opposite edge gradient of the pixel point of the edge to be determined;
step 2.4, calculating the probability that the current pixel point is an edge pixel point according to the gray value, the shape edge strength and the relative edge gradient of the current pixel point;
step 2.5, binarizing the edge feature map according to gamma (p), and obtaining n binarized shape edge feature maps corresponding to each operator size;
step 2.6, performing OR operation on n binarized shape edge feature graphs corresponding to each operator size, and fusing the n binarized shape edge feature graphs into a polygonal edge shape feature graph E S A multi-shape edge feature map set including m multi-shape edge feature maps is obtained.
4. The unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation according to claim 3, wherein the binarization operation specifically comprises: regarding the pixel points with gamma (p) higher than the set threshold value as edge pixel points, and setting the gray value as 255; the pixel below the threshold is considered as a non-edge pixel, and the gray value is set to 0.
5. The unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation according to claim 3, wherein the shape edge intensity of the pixel point pOpposite edge gradient->Wherein Gray (p-u) represents the Gray value of the u-th pixel in the negative direction of the edge shape of the pixel p, gray (p+u) represents the Gray value of the u-th pixel in the positive direction of the edge shape of the pixel p, and Gray (p) represents the Gray value of the pixel p.
6. The unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation according to claim 3, wherein the probability that the current pixel point p is an edge pixel point isGray(p)、/>Respectively representing the gray value of the current pixel, the shape edge intensity and the opposite edge gradient,representing the average gray value, average shape edge intensity and average relative edge gradient of the pixel points of the undetermined edge respectively.
7. The unmanned aerial vehicle cruise detection method based on the adaptive edge feature semantic segmentation according to claim 1, wherein the multi-scale edge feature weighting method based on the target matching degree in the step 3 comprises the following steps:
step 3.1, defining a height influence factor, wherein the specific calculation method comprises the following steps:
wherein [ Alt ] min ,Alt max ]Represents the unmanned aerial vehicle cruising altitude interval, alt cur Representing the altitude at which the inspection image of the drone was acquired,represents the cruising average height, k of the unmanned aerial vehicle i 、k j Respectively representing the operator scales corresponding to the ith and jth multi-shape edge feature images in the multi-shape edge feature image group, < ->Representing the height influence factor corresponding to the ith multi-shape edge feature map in the multi-shape edge feature map set;
defining an edge quantity factorThe specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,represents an intermediate variable,/->Representing the number of edge points in the ith and jth polygonal edge shape feature images in the multi-shape edge feature image group;
and 3.2, calculating the target matching degree according to the height influence factor and the edge quantity factor obtained in the step 3.1, wherein the specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the target matching degree corresponding to the ith polygonal edge shape feature map in the multi-shape edge feature map group;
and 3.3, weighting the polygonal edge shape feature image group according to the target matching degree to obtain a multi-scale multi-shape edge feature image group, wherein the specific calculation method is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,weighting the i-th multi-scale multi-shape edge feature map in the multi-shape edge feature map set, ++>Is the ith polygon edge shape characteristic diagram in the multi-shape edge characteristic diagram group.
8. The unmanned aerial vehicle cruise detection method based on the self-adaptive edge feature semantic segmentation according to claim 1, wherein an improved deep LabV3+ semantic segmentation network is built in the step 4, and the method specifically comprises the following steps:
step 4.1, using a receptive field fusion cavity space pyramid pooling module at an encoder part of the deep LabV3+ semantic segmentation network to carry out weighted fusion on branch self-adaption with different receptive fields, wherein the construction process of the receptive field fusion cavity space pyramid pooling module is as follows:
step 4.1.1 for convolutional layer P 1 And 3 hole convolution layers P 2 、P 3 、P 4 Splicing operation is carried out in the channel direction to obtain fusion characteristicsWherein (1)>Representing three-dimensional features of the feature map, H representing feature map height, W representing feature map width, and C representing feature map depth;
step 4.1.2, pooling the fusion feature P in the height direction and the width direction to obtainAnd
step 4.1.3 for Z h And Z w Performing channel splicing operation, adjusting channel number by using 1×1 convolution operation, activating by using a ReLU function to obtainWherein r is a scaling factor;
step 4.1.4 splitting f into two independent featuresAnd->Then the 1X 1 convolution operation is used for carrying out the dimension lifting operation, and the Sigmoid function is used for activating, so that +.>And->
Step 4.1.5, using g h And g w Calculating a weighted output P 'and pooling the weighted output P' with the global average pooling layer output P 5 Performing channel splicing operation to obtain the integral output P' of the receptive field fusion cavity space pyramid pooling module;
and 4.2, in a decoder part of the deep LabV3+ semantic segmentation network, performing four-time downsampling on the multi-scale multi-shape edge feature image group, and then splicing the multi-scale multi-shape edge feature image group with the advanced semantic features obtained by the encoder part in the channel direction.
9. Unmanned aerial vehicle cruise detection system based on self-adaptive edge feature semantic segmentation, which is characterized by carrying out unmanned aerial vehicle cruise detection based on the method according to any one of claims 1 to 7, and specifically comprises the following steps:
the multi-shape edge feature fusion module is used for obtaining a multi-shape edge feature image group according to a predefined multi-scale multi-shape edge detection operator group;
the multi-scale edge feature weighting module is used for weighting according to the target matching degree to obtain a multi-scale multi-shape edge feature image group;
the semantic segmentation network module is used for building a semantic segmentation network and training.
10. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the method of any of claims 1-7.
CN202310471491.7A 2023-04-27 2023-04-27 Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation Pending CN116503760A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310471491.7A CN116503760A (en) 2023-04-27 2023-04-27 Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310471491.7A CN116503760A (en) 2023-04-27 2023-04-27 Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation

Publications (1)

Publication Number Publication Date
CN116503760A true CN116503760A (en) 2023-07-28

Family

ID=87322614

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310471491.7A Pending CN116503760A (en) 2023-04-27 2023-04-27 Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation

Country Status (1)

Country Link
CN (1) CN116503760A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116894775A (en) * 2023-09-11 2023-10-17 中铁大桥局集团第二工程有限公司 Bolt image preprocessing method based on camera motion model recovery and super-resolution

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116894775A (en) * 2023-09-11 2023-10-17 中铁大桥局集团第二工程有限公司 Bolt image preprocessing method based on camera motion model recovery and super-resolution
CN116894775B (en) * 2023-09-11 2024-01-23 中铁大桥局集团第二工程有限公司 Bolt image preprocessing method based on camera motion model recovery and super-resolution

Similar Documents

Publication Publication Date Title
CN106780485B (en) SAR image change detection method based on super-pixel segmentation and feature learning
CN111340797A (en) Laser radar and binocular camera data fusion detection method and system
CN113705478B (en) Mangrove single wood target detection method based on improved YOLOv5
Siriborvornratanakul An automatic road distress visual inspection system using an onboard in-car camera
CN109977997B (en) Image target detection and segmentation method based on convolutional neural network rapid robustness
Li et al. Automatic bridge crack identification from concrete surface using ResNeXt with postprocessing
CN112287832A (en) High-resolution remote sensing image-based urban illegal building detection method
CN116402821B (en) Aircraft skin gluing quality defect detection method based on neural network
CN114049356B (en) Method, device and system for detecting structure apparent crack
CN110827312A (en) Learning method based on cooperative visual attention neural network
CN115049821A (en) Three-dimensional environment target detection method based on multi-sensor fusion
CN110909623A (en) Three-dimensional target detection method and three-dimensional target detector
CN116503760A (en) Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation
CN112364881A (en) Advanced sampling consistency image matching algorithm
CN113989604A (en) Tire DOT information identification method based on end-to-end deep learning
CN113269147A (en) Three-dimensional detection method and system based on space and shape, and storage and processing device
CN117496401A (en) Full-automatic identification and tracking method for oval target points of video measurement image sequences
CN116310902A (en) Unmanned aerial vehicle target detection method and system based on lightweight neural network
CN114005110B (en) 3D detection model training method and device, and 3D detection method and device
Dong et al. Fast infrared horizon detection algorithm based on gradient directional filtration
Guo et al. River Extraction Method of Remote Sensing Image Based on Edge Feature Fusion
CN114463628A (en) Deep learning remote sensing image ship target identification method based on threshold value constraint
CN113724329A (en) Object attitude estimation method, system and medium fusing plane and stereo information
Ashraf et al. Efficient Pavement Crack Detection and Classification Using Custom YOLOv7 Model
Chen et al. Learning rotation sensitive neural network for deformed objects' detection in fisheye images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination