CN117726958B - Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line - Google Patents
Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line Download PDFInfo
- Publication number
- CN117726958B CN117726958B CN202410171362.0A CN202410171362A CN117726958B CN 117726958 B CN117726958 B CN 117726958B CN 202410171362 A CN202410171362 A CN 202410171362A CN 117726958 B CN117726958 B CN 117726958B
- Authority
- CN
- China
- Prior art keywords
- image
- feature
- distribution line
- aerial vehicle
- unmanned aerial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000001514 detection method Methods 0.000 title claims abstract description 21
- 238000007689 inspection Methods 0.000 title claims abstract description 18
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 230000003287 optical effect Effects 0.000 claims abstract description 4
- 230000008447 perception Effects 0.000 claims abstract description 4
- 230000004913 activation Effects 0.000 claims description 24
- 230000004927 fusion Effects 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 15
- 238000011176 pooling Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000007670 refining Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000007500 overflow downdraw method Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 4
- 230000002457 bidirectional effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 150000001875 compounds Chemical class 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000003706 image smoothing Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims 1
- 238000012423 maintenance Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 230000009471 action Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application relates to a distribution line unmanned aerial vehicle inspection image target detection and hidden danger intelligent identification method, which comprises the following steps: the unmanned aerial vehicle is used for collecting optical images and infrared images of the distribution line, and preprocessing infrared images and visible images of the unmanned aerial vehicle collected on the distribution line; constructing a typical target detection model under a complex background driven by large model semantics, reading the preprocessed image, and extracting the characteristics of the image; and constructing a distribution line hidden danger positioning and fine granularity identification algorithm combining the perception attention, and carrying out fine granularity hidden danger identification on the acquired characteristics. The application can realize comprehensive monitoring of the distribution line, quickly locate hidden trouble, accurately plan maintenance plan, improve maintenance efficiency and reduce maintenance cost through an automatic hidden trouble identification technology.
Description
Technical Field
The application relates to the field of distribution line hidden danger identification and detection, in particular to a distribution line unmanned aerial vehicle inspection image target detection and hidden danger intelligent identification method.
Background
In the past, the potential safety hazard of distribution lines is eliminated by adopting a mode of manually observing and shooting by a camera along the distribution lines by professional inspection staff, and the mode is low in efficiency and high in manpower cost, and the detection result is very dependent on the experience of the professional staff. Workers are also at great risk from the standpoint of personal safety. If the situation of severe environment happens, the running cost of inspection is greatly increased.
Disclosure of Invention
The embodiment of the application aims to provide a distribution line unmanned aerial vehicle inspection image target detection and hidden danger intelligent identification method, which is used for solving the three core problems of low sample quality, insufficient sample quantity, large typical target scale difference and difficult identification of fine-granularity hidden danger existing in the distribution line unmanned aerial vehicle inspection process.
In order to achieve the above purpose, the present application provides the following technical solutions:
The embodiment of the application provides a method for detecting a patrol image target of a power distribution line unmanned aerial vehicle and intelligently identifying hidden danger, which comprises the following steps:
the unmanned aerial vehicle is used for collecting optical images and infrared images of the distribution line, and preprocessing infrared images and visible images of the unmanned aerial vehicle collected on the distribution line;
constructing a typical target detection model under a complex background driven by large model semantics, reading the preprocessed image, and extracting the characteristics of the image;
And constructing a distribution line hidden danger positioning and fine granularity identification algorithm combining the perception attention, and carrying out fine granularity hidden danger identification on the acquired characteristics.
The preprocessing of the unmanned aerial vehicle infrared and visible light images collected on the distribution line is specifically that details of cross-domain images are added by a sample refining and image enhancement method based on a residual pyramid network, in the process of the sample refining and image enhancement method based on the residual pyramid network, a multi-type sample refining method is adopted, and 5 types of image enhancement processing including noise adding, blurring processing, edge sharpening, pixel inversion and image smoothing are performed on an original image, so that the problem of low sample capacity is solved, and the data size is further amplified.
In the residual pyramid network-based sample refinement and image enhancement method, feature extraction is performed on different layers of information by using a pyramid cascade network, and a calculation formula is as follows:
，
In the above-mentioned description of the invention, Representing feature splice sets generated by the first layer dense residual block, the second layer dense residual block and the Mth layer dense residual block respectively,/>A convolution operation representing 1*1 of layer h, global residual learning in network/>Is defined as:
，
In the above-mentioned formula, the group of the compounds, Representing the/>, in a low resolution subnetworkLayer output, output in high resolution subnetworkAs an input to the low resolution network.
In the process of the sample refinement and image enhancement method based on the residual pyramid network, a visible light and infrared image fusion method of a combined full convolution neural network is adopted, a camera and an infrared sensor are installed through an unmanned aerial vehicle, and the visible light and the infrared image are simultaneously acquired and subjected to later fusion processing so as to reflect imaging information of a distribution line image more truly; in the process of a visible light and infrared image fusion method adopting a combined full convolution neural network, a low-frequency image fusion mode and a high-frequency image fusion mode are introduced, so that the resolution and the definition of a fused image are ensured, and the accuracy of circuit line inspection fault detection is improved.
The method for constructing the typical target detection model under the complex background driven by the large model semanteme, reading the preprocessed image, and extracting the characteristics of the image specifically comprises the following steps of:
Dry sampling function consisting of two convolution layers Scaling the features to a suitable size, each convolution layer having a convolution kernel size of 3, a stride of 2, and a fill of 1, including an LN normalization layer and a GELU activation function layer;
Performing DCNv function processing on the scaled features;
Inserting a downsampling layer to adjust the size of the feature map, filtering redundant information and noise in the data, and retaining key features of the data;
based on the designed superimposition criteria, an image is output.
The multi-scale feature extraction for infrared images is as follows:
Slicing the data through a Focus structure, sampling at intervals to obtain 4 feature graphs, and expanding the number of input channels by 4 times;
after one convolution operation, the data is input to Module/>The module realizes data dimension reduction by adjusting the channel number, reduces the network parameter number, and then adopts point convolution to adjust the channel number to be consistent with input so as to realize channel information fusion;
After one convolution operation, the data is transmitted to the SPP layer, the SPP layer separates the context features by convolution and pooling operation, and the receptive field is increased, so that the subsequent fusion of global feature information is facilitated.
The method for fusing the visible light and the infrared features by using a multi-mode bidirectional feature fusion algorithm comprises the following specific steps:
An original feature pyramid is obtained by carrying out a series of convolution operations on an input image, and feature graphs with different scales are obtained;
After obtaining feature graphs with different scales, combining the feature graphs to form an initial multi-scale feature pyramid, wherein the feature graphs have the characteristics of high-to-low resolution and high-to-low semantic information;
transmitting shallow characteristic information to a deep layer through a bottom-up structure, simultaneously downsampling high-resolution characteristics, and carrying out channel transformation on the characteristic map to obtain a new characteristic map;
through reverse feature transfer, semantic features in the deep feature map are transferred to the shallow feature map, and a new multi-scale feature map is generated;
and combining all the enhanced feature graphs with different scales to form a new feature pyramid, so as to finish the fusion of the visible light and infrared features.
Target detection using the trifoliate-based attention is performed as follows:
The feature tensor is subjected to global average pooling, global maximum pooling and global random pooling of three parallel modes to obtain three channel feature quantities with C multiplied by 1 dimension,
Adding the generated three channel characteristic quantities to obtain a comprehensive channel characteristic quantity with the dimension of Cx1x1,
Nonlinear activation is performed by using a Sigmoid function to obtain a channel weight coefficient of the feature map, the input feature map is multiplied by the channel weight coefficient and is connected with the input feature map in a jumping manner to obtain a brand new feature map with channel importance difference,
And adding the three-way dimensionality halved output feature images, inputting the feature images into a Sigmoid activation function to obtain a final spatial attention weight matrix, and multiplying the weight matrix by the input feature images to obtain the feature images with spatial pixel weight relations.
The method for locating the image local discrimination area under the guidance of the attention introducing mechanism specifically comprises the following steps of:
Giving a global image, taking an absolute value of an activation value at any position, and obtaining a visual attention heat map of the feature by taking the maximum value in the direction of the feature channel;
designing a binary template for locating a local area with a larger activation value in the heat map;
acquiring a maximum communication area capable of covering all larger activation values in the binary template from the characteristic heat map;
And cutting out a local judging region in the input image by utilizing the maximum connected region, and then adjusting the local judging region to be the same size as the global image.
Compared with the prior art, the application has the beneficial effects that:
By means of the accurate hidden danger identification technology, potential problems can be found early, preventive and maintenance measures are taken, accident risks are reduced, and the overall safety level of the power system is improved. The hidden danger identification technology can immediately find problems before the distribution line is in problem, avoid interruption and faults of the line and ensure the continuity and stability of power supply. By maintaining and replacing damaged parts in time, the power failure time can be reduced, the power utilization satisfaction of users can be improved, and the reliable power supply of a power system can be ensured. The traditional line inspection method requires a lot of manpower and time, and is low in efficiency and easy to miss. Through automatic hidden danger identification technology, can realize the comprehensive monitoring to distribution lines, the hidden danger is fixed a position fast, accurate planning maintenance plan improves maintenance efficiency, reduces maintenance cost. With the increasing demand for electricity and the increasing upgrading of the power grid, the requirements for safety and reliability of the distribution lines are increasing. The application of the distribution line hidden trouble identification technology is helpful to promote the power industry to develop towards the direction of intellectualization and digitalization, promote the overall level of the power system and meet the requirements of future energy supply. The invention has the advantages of ensuring safe and stable operation of the power system, improving the reliability of power supply, optimizing maintenance and management of facilities, providing support for future development of the power industry and promoting the power industry to realize sustainable development and modernization transformation.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The terms "first," "second," and the like, are used merely to distinguish one entity or action from another entity or action, and are not to be construed as indicating or implying any actual such relationship or order between such entities or actions.
Referring to fig. 1, an embodiment of the invention provides a method for detecting a target of an inspection image of a power distribution line unmanned aerial vehicle and intelligently identifying hidden trouble, which comprises the following steps:
S1, acquiring optical images and infrared images of a distribution line by using an unmanned aerial vehicle, and preprocessing infrared and visible light images of the unmanned aerial vehicle acquired on the distribution line;
S2, constructing a typical target detection model under a complex background driven by large model semantics, reading the preprocessed image, and extracting the characteristics of the image;
s3, constructing a distribution line hidden danger positioning and fine granularity identification algorithm combining the perception attention, and carrying out fine granularity hidden danger identification on the acquired characteristics.
The preprocessing of the unmanned aerial vehicle infrared and visible light images collected on the distribution line is specifically that details of cross-domain images are added by a sample refining and image enhancement method based on a residual pyramid network, in the process of the sample refining and image enhancement method based on the residual pyramid network, a multi-type sample refining method is adopted, and 5 types of image enhancement processing including noise adding, blurring processing, edge sharpening, pixel inversion and image smoothing are performed on an original image, so that the problem of low sample capacity is solved, and the data size is further amplified.
In the residual pyramid network-based sample refinement and image enhancement method, feature extraction is performed on different layers of information by using a pyramid cascade network, and a calculation formula is as follows:
，
In the above-mentioned description of the invention, Representing feature splice sets generated by the first layer dense residual block, the second layer dense residual block and the Mth layer dense residual block respectively,/>A convolution operation representing 1*1 of layer h, global residual learning in network/>Is defined as:
，
In the above-mentioned formula, the group of the compounds, Representing the/>, in a low resolution subnetworkLayer output, output in high resolution subnetworkAs an input to the low resolution network.
In the process of the sample refinement and image enhancement method based on the residual pyramid network, a visible light and infrared image fusion method of a combined full convolution neural network is adopted, a camera and an infrared sensor are installed through an unmanned aerial vehicle, and the visible light and the infrared image are simultaneously acquired and subjected to later fusion processing so as to reflect imaging information of a distribution line image more truly; in the process of a visible light and infrared image fusion method adopting a combined full convolution neural network, a low-frequency image fusion mode and a high-frequency image fusion mode are introduced, so that the resolution and the definition of a fused image are ensured, and the accuracy of circuit line inspection fault detection is improved.
The method for constructing the typical target detection model under the complex background driven by the large model semanteme, reading the preprocessed image, and extracting the characteristics of the image specifically comprises the following steps of:
Dry sampling function consisting of two convolution layers Scaling the features to a suitable size, each convolution layer having a convolution kernel size of 3, a stride of 2, and a fill of 1, including an LN normalization layer and a GELU activation function layer;
Performing DCNv function processing on the scaled features;
Inserting a downsampling layer to adjust the size of the feature map, filtering redundant information and noise in the data, and retaining key features of the data;
based on the designed superimposition criteria, an image is output.
The multi-scale feature extraction for infrared images is as follows:
Slicing the data through a Focus structure, sampling at intervals to obtain 4 feature graphs, and expanding the number of input channels by 4 times;
after one convolution operation, the data is input to Module/>The module realizes data dimension reduction by adjusting the channel number, reduces the network parameter number, and then adopts point convolution to adjust the channel number to be consistent with input so as to realize channel information fusion;
After one convolution operation, the data is transmitted to the SPP layer, the SPP layer separates the context features by convolution and pooling operation, and the receptive field is increased, so that the subsequent fusion of global feature information is facilitated.
The method for fusing the visible light and the infrared features by using a multi-mode bidirectional feature fusion algorithm comprises the following specific steps:
An original feature pyramid is obtained by carrying out a series of convolution operations on an input image, and feature graphs with different scales are obtained;
After obtaining feature graphs with different scales, combining the feature graphs to form an initial multi-scale feature pyramid, wherein the feature graphs have the characteristics of high-to-low resolution and high-to-low semantic information;
transmitting shallow characteristic information to a deep layer through a bottom-up structure, simultaneously downsampling high-resolution characteristics, and carrying out channel transformation on the characteristic map to obtain a new characteristic map;
through reverse feature transfer, semantic features in the deep feature map are transferred to the shallow feature map, and a new multi-scale feature map is generated;
and combining all the enhanced feature graphs with different scales to form a new feature pyramid, so as to finish the fusion of the visible light and infrared features.
Target detection using the trifoliate-based attention is performed as follows:
The feature tensor is subjected to global average pooling, global maximum pooling and global random pooling of three parallel modes to obtain three channel feature quantities with C multiplied by 1 dimension,
Adding the generated three channel characteristic quantities to obtain a comprehensive channel characteristic quantity with the dimension of Cx1x1,
Nonlinear activation is performed by using a Sigmoid function to obtain a channel weight coefficient of the feature map, the input feature map is multiplied by the channel weight coefficient and is connected with the input feature map in a jumping manner to obtain a brand new feature map with channel importance difference,
And adding the three-way dimensionality halved output feature images, inputting the feature images into a Sigmoid activation function to obtain a final spatial attention weight matrix, and multiplying the weight matrix by the input feature images to obtain the feature images with spatial pixel weight relations.
The image local discrimination area positioning method under the guidance of the attention mechanism constructs a binary mask for positioning the local discrimination area in the global image. The binary template has the same size as the global image, and each position value is 0 or 1,1 is the local discrimination area, and 0 is the non-discrimination area. The binary template is obtained by setting a threshold in the feature map (feature maps). This process can be regarded as a visually noticeable process. Specific method steps are given below.
First, given a global image I, setIndicating that its last convolutional layer in the global branch is in space/>Location, th/>Activation value of individual channel,/>，/>The total number of channels characterizing the last convolutional layer in the global branch, as in ResNet-50,/>。/>Representing a global branch. First for any positionActivation value/>Taking the absolute value. A visual attention heat map (attention heatmap)/>, of the feature is then obtained by taking the maximum value in the feature channel direction。
The magnitude of the value directly reflects/>The importance of the location features to the classification of hidden danger.
The image local discrimination area positioning method under the guidance of the attention mechanism designs a binary template for positioning a local area with larger activation value in the heat map. First, in a feature heat mapIf any position/>The activation value in the heat map is greater than a predetermined threshold/>The corresponding position value in the template is set to 1, otherwise to 0. In particular to
Wherein,For controlling the size of the attention area. /(I)The larger the acquired local area is, the smaller the local area is. /(I)The smaller the acquired local area is, the larger the acquired local area is. In extreme cases, if/>Is 0 due to/>The templates are all 1, and the local image and the global image acquired at this time are the same. If/>1, Due to/>The templates are all 0's, and at this time, the local discrimination area cannot be obtained. Therefore, it is set as/>. In practice, there may be a feature map containing multiple regions of greater activation values, which may be connected or separated. In order to cover all areas with larger activation values, a maximum connected area which can cover all areas with larger activation values in the template is acquired on the characteristic heat map by using a related method. The maximum communication area position information is expressed as:
Wherein, ，/>The upper left and lower right corner coordinates of the maximum communication area in the feature map, respectively.
Finally, the local discrimination area is cut out in the input image I by utilizing RAnd then/>And adjusting the size to be equal to the I. By using the image local discriminant region localization method under the guidance of the attention mechanism, the region or feature of interest in the image can be precisely located without processing the entire image. This helps to improve positioning accuracy and efficiency.
For the original input image, it can be considered a global scale image. Using the foreground attention awareness mechanism, the foreground image may be obtained from the global image without using bounding box annotations. In order to extract the discriminative part area without using part labeling or bounding boxes, the last feature map of the foreground scale feature extractor is put into a part attention sensing mechanism, and then a part image with discriminative features can be obtained. The three scale images are placed in their feature decimators to obtain three logits. When carrying out reasoning prediction, the IAPP-CNN algorithm integrates the logit output by three CNN branches to obtain the final prediction category, which is different from the former work. Furthermore, due to the risk of identified uncertainties, a joint uncertainty estimation module is introduced that evaluates their reliability before each log addition. The introduction of the combined uncertainty estimation module further improves the accuracy and the robustness of hidden danger identification. The main idea of IAPP-CNN is to construct a new deep neural network, mine more information from fine-grained images, and make more robust predictions for fine-grained recognition.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and variations will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (5)
1. The intelligent detection and hidden danger identification method for the inspection image target of the unmanned aerial vehicle of the power distribution line is characterized by comprising the following steps:
the unmanned aerial vehicle is used for collecting optical images and infrared images of the distribution line, and preprocessing infrared images and visible images of the unmanned aerial vehicle collected on the distribution line;
constructing a typical target detection model under a complex background driven by large model semantics, reading the preprocessed image, and extracting the characteristics of the image;
Constructing a distribution line hidden danger positioning and fine granularity identifying algorithm combining the perception attention, and identifying the fine granularity hidden danger of the acquired characteristics;
The preprocessing of the unmanned aerial vehicle infrared and visible light images acquired on the distribution line comprises the steps of adding details of cross-domain images by a residual pyramid network-based sample refining and image enhancement method, adopting a multi-type sample refining method in the process of the residual pyramid network-based sample refining and image enhancement method, adding noise, blurring, edge sharpening, pixel inversion and image smoothing 5 types of image enhancement processing to the original image, solving the problem of low sample capacity, and further amplifying data volume
In the residual pyramid network-based sample refinement and image enhancement method, feature extraction is performed on different layers of information by using a pyramid cascade network, and a calculation formula is as follows:
，
In the above-mentioned description of the invention, Representing feature splice sets generated by the first layer dense residual block, the second layer dense residual block and the Mth layer dense residual block respectively,/>A convolution operation representing 1*1 of layer h, global residual learning in network/>Is defined as:
，
In the above-mentioned formula, the group of the compounds, Representing the/>, in a low resolution subnetworkLayer output, output in high resolution subnetworkAs input to a low resolution network;
In the process of the sample refinement and image enhancement method based on the residual pyramid network, a visible light and infrared image fusion method of a combined full convolution neural network is adopted, a camera and an infrared sensor are installed through an unmanned aerial vehicle, and the visible light and the infrared image are simultaneously acquired and subjected to later fusion processing so as to reflect imaging information of a distribution line image more truly; in the process of a visible light and infrared image fusion method combining a full convolution neural network, a low-frequency image fusion mode and a high-frequency image fusion mode are introduced, so that the resolution and the definition of a fused image are ensured, and the accuracy of circuit line inspection fault detection is improved;
The method for locating the image local discrimination area under the guidance of the attention introducing mechanism specifically comprises the following steps of:
First, given a global image I, set Indicating that its last convolutional layer in the global branch is in spaceLocation, th/>Activation value of individual channel,/>，/>Total number of channels representing the last convolutional layer feature in a global branch,/>Representing global branches, first for arbitrary locations/>Activation value/>Taking absolute value and then obtaining the visual attention heat map/>, of the feature by taking the maximum value in the direction of the feature channel，
，
The magnitude of the value directly reflects/>The importance of the location features to the classification of the hidden trouble,
The image local discrimination area positioning method under the guidance of the attention mechanism designs a binary template for positioning a local area with larger activation value in a heat map, firstly, in a characteristic heat mapIf any position/>The activation value in the heat map is greater than a predetermined threshold/>The corresponding position value in the template is set to 1, otherwise, is set to 0, in particular
，
Wherein,For controlling the size of the attention area,/>The larger the local region acquired, the smaller the/>The smaller the acquired local area the larger, in extreme cases if/>Is 0 due to/>The templates are all 1, the local image and the global image acquired at the moment are the same, if/>1, Due to/>The templates are all 0, and the local discrimination area cannot be obtained at this time, and therefore, it is set to/>In order to cover all areas with larger activation values, a maximum communication area which can cover all areas with larger activation values in a template is acquired on a characteristic heat map by using a related method, and the position information of the maximum communication area is expressed as:
，
Wherein, ，/>The upper left and lower right corner coordinates of the maximum communication area in the feature map,
Finally, the local discrimination area is cut out in the input image I by utilizing RAnd then/>The method is adjusted to be of the same size, and by using the image local discrimination area positioning method under the guidance of the attention mechanism, the region or feature of interest in the image can be accurately positioned without processing the whole image, which is helpful to improve positioning accuracy and efficiency.
2. The method for detecting the target of the inspection image of the unmanned aerial vehicle of the power distribution line and intelligently identifying hidden danger according to claim 1, wherein the construction of a typical target detection model under a complex background driven by large model semantics, reading the preprocessed image, and extracting the characteristics of the image, specifically, for the input visible light image characteristics, comprises the following steps:
Dry sampling function consisting of two convolution layers Scaling the features to a suitable size, each convolution layer having a convolution kernel size of 3, a stride of 2, and a fill of 1, including an LN normalization layer and a GELU activation function layer;
Performing DCNv function processing on the scaled features;
Inserting a downsampling layer to adjust the size of the feature map, filtering redundant information and noise in the data, and retaining key features of the data;
based on the designed superimposition criteria, an image is output.
3. The method for detecting the target of the inspection image of the unmanned aerial vehicle for the power distribution line and intelligently identifying hidden dangers according to claim 2, wherein the steps for extracting the multi-scale characteristics of the infrared image are as follows:
Slicing the data through a Focus structure, sampling at intervals to obtain 4 feature graphs, and expanding the number of input channels by 4 times;
after one convolution operation, the data is input to Module/>The module realizes data dimension reduction by adjusting the channel number, reduces the network parameter number, and then adopts point convolution to adjust the channel number to be consistent with input so as to realize channel information fusion;
After one convolution operation, the data is transmitted to the SPP layer, the SPP layer separates the context features by convolution and pooling operation, and the receptive field is increased, so that the subsequent fusion of global feature information is facilitated.
4. The method for detecting the inspection image target and intelligently identifying hidden danger of the unmanned aerial vehicle of the power distribution line according to claim 3, wherein the method for fusing the visible light and the infrared features by using a multi-mode bidirectional feature fusion algorithm comprises the following specific steps:
An original feature pyramid is obtained by carrying out a series of convolution operations on an input image, and feature graphs with different scales are obtained;
After obtaining feature graphs with different scales, combining the feature graphs to form an initial multi-scale feature pyramid, wherein the feature graphs have the characteristics of high-to-low resolution and high-to-low semantic information;
transmitting shallow characteristic information to a deep layer through a bottom-up structure, simultaneously downsampling high-resolution characteristics, and carrying out channel transformation on the characteristic map to obtain a new characteristic map;
through reverse feature transfer, semantic features in the deep feature map are transferred to the shallow feature map, and a new multi-scale feature map is generated;
and combining all the enhanced feature graphs with different scales to form a new feature pyramid, so as to finish the fusion of the visible light and infrared features.
5. The method for detecting the target of the inspection image and intelligently identifying hidden dangers of the unmanned aerial vehicle for the power distribution line according to claim 4, wherein the target detection based on the attention of the trigeminal anvils is used, and comprises the following specific steps:
The feature tensor is subjected to global average pooling, global maximum pooling and global random pooling of three parallel modes to obtain three channel feature quantities with C multiplied by 1 dimension,
Adding the generated three channel characteristic quantities to obtain a comprehensive channel characteristic quantity with the dimension of Cx1x1,
Nonlinear activation is performed by using a Sigmoid function to obtain a channel weight coefficient of the feature map, the input feature map is multiplied by the channel weight coefficient and is connected with the input feature map in a jumping manner to obtain a brand new feature map with channel importance difference,
And adding the three-way dimensionality halved output feature images, inputting the feature images into a Sigmoid activation function to obtain a final spatial attention weight matrix, and multiplying the weight matrix by the input feature images to obtain the feature images with spatial pixel weight relations.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410171362.0A CN117726958B (en) | 2024-02-07 | 2024-02-07 | Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410171362.0A CN117726958B (en) | 2024-02-07 | 2024-02-07 | Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117726958A CN117726958A (en) | 2024-03-19 |
| CN117726958B true CN117726958B (en) | 2024-05-10 |
Family
ID=90200103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410171362.0A Active CN117726958B (en) | 2024-02-07 | 2024-02-07 | Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117726958B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118011152A (en) * | 2024-04-08 | 2024-05-10 | 陕西公众电气股份有限公司 | Distribution line hidden danger identification system and method and electronic equipment |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188705A (en) * | 2019-06-02 | 2019-08-30 | 东北石油大学 | A kind of remote road traffic sign detection recognition methods suitable for onboard system |
| CN112131936A (en) * | 2020-08-13 | 2020-12-25 | 华瑞新智科技（北京）有限公司 | Inspection robot image identification method and inspection robot |
| CN112163465A (en) * | 2020-09-11 | 2021-01-01 | 华南理工大学 | Fine-grained image classification method, fine-grained image classification system, computer equipment and storage medium |
| CN112784869A (en) * | 2020-11-13 | 2021-05-11 | 北京航空航天大学 | Fine-grained image identification method based on attention perception and counterstudy |
| CN114241386A (en) * | 2021-12-21 | 2022-03-25 | 江苏翰林正川工程技术有限公司 | Method for detecting and identifying hidden danger of power transmission line based on real-time video stream |
| CN115471723A (en) * | 2022-09-23 | 2022-12-13 | 安徽优航遥感信息技术有限公司 | Substation unmanned aerial vehicle inspection method based on infrared and visible light image fusion |
| CN116563781A (en) * | 2023-04-25 | 2023-08-08 | 广西电网有限责任公司来宾供电局 | Image monitoring and diagnosing method for inspection robot |
| CN116863285A (en) * | 2023-07-10 | 2023-10-10 | 兰州交通大学 | Infrared and visible light image fusion method for multiscale generation countermeasure network |
| CN116958782A (en) * | 2023-07-05 | 2023-10-27 | 中国电子科技集团公司第十五研究所 | Method and device for detecting weak and small targets by combining infrared and visible light characteristics |
| CN117274899A (en) * | 2023-09-20 | 2023-12-22 | 中国人民解放军海军航空大学 | Storage hidden danger detection method based on visible light and infrared light image feature fusion |
| CN117350964A (en) * | 2023-10-07 | 2024-01-05 | 安徽大学 | Cross-modal multi-level feature fusion-based power equipment detection method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111433810A (en) * | 2018-12-04 | 2020-07-17 | 深圳市大疆创新科技有限公司 | Target image acquisition method, shooting device and unmanned aerial vehicle |
-
2024
- 2024-02-07 CN CN202410171362.0A patent/CN117726958B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188705A (en) * | 2019-06-02 | 2019-08-30 | 东北石油大学 | A kind of remote road traffic sign detection recognition methods suitable for onboard system |
| CN112131936A (en) * | 2020-08-13 | 2020-12-25 | 华瑞新智科技（北京）有限公司 | Inspection robot image identification method and inspection robot |
| CN112163465A (en) * | 2020-09-11 | 2021-01-01 | 华南理工大学 | Fine-grained image classification method, fine-grained image classification system, computer equipment and storage medium |
| CN112784869A (en) * | 2020-11-13 | 2021-05-11 | 北京航空航天大学 | Fine-grained image identification method based on attention perception and counterstudy |
| CN114241386A (en) * | 2021-12-21 | 2022-03-25 | 江苏翰林正川工程技术有限公司 | Method for detecting and identifying hidden danger of power transmission line based on real-time video stream |
| CN115471723A (en) * | 2022-09-23 | 2022-12-13 | 安徽优航遥感信息技术有限公司 | Substation unmanned aerial vehicle inspection method based on infrared and visible light image fusion |
| CN116563781A (en) * | 2023-04-25 | 2023-08-08 | 广西电网有限责任公司来宾供电局 | Image monitoring and diagnosing method for inspection robot |
| CN116958782A (en) * | 2023-07-05 | 2023-10-27 | 中国电子科技集团公司第十五研究所 | Method and device for detecting weak and small targets by combining infrared and visible light characteristics |
| CN116863285A (en) * | 2023-07-10 | 2023-10-10 | 兰州交通大学 | Infrared and visible light image fusion method for multiscale generation countermeasure network |
| CN117274899A (en) * | 2023-09-20 | 2023-12-22 | 中国人民解放军海军航空大学 | Storage hidden danger detection method based on visible light and infrared light image feature fusion |
| CN117350964A (en) * | 2023-10-07 | 2024-01-05 | 安徽大学 | Cross-modal multi-level feature fusion-based power equipment detection method |
Non-Patent Citations (2)
| Title |
|---|
| 基于卷积神经网络的毫米波图像人体隐匿物检测;骆尚;吴晓峰;杨明辉;王斌;孙晓玮;;复旦学报(自然科学版);20180815(第04期);全文 * |
| 基于可见光、热红外及激光雷达传感的无人机图像融合方法;汪勇;张英;廖如超;郭启迪;袁新星;康泰钟;;激光杂志;20200225(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117726958A (en) | 2024-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Qiu et al. | Automatic visual defects inspection of wind turbine blades via YOLO-based small object detection approach | |
| CN117726958B (en) | Intelligent detection and hidden danger identification method for inspection image target of unmanned aerial vehicle of distribution line | |
| CN109977921B (en) | Method for detecting hidden danger of power transmission line | |
| CN109858367B (en) | Visual automatic detection method and system for worker through supporting unsafe behaviors | |
| CN108711148A (en) | A kind of wheel tyre defect intelligent detecting method based on deep learning | |
| CN114049356B (en) | Method, device and system for detecting structure apparent crack | |
| CN115719475B (en) | Three-stage trackside equipment fault automatic detection method based on deep learning | |
| CN116645616A (en) | Method for evaluating damage degree of building in natural disasters by using remote sensing images | |
| CN112949457A (en) | Maintenance method, device and system based on augmented reality technology | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN116385421A (en) | Photovoltaic panel detection method, unmanned aerial vehicle and computer readable storage medium | |
| CN117252842A (en) | Aircraft skin defect detection and network model training method | |
| CN115147439A (en) | Concrete crack segmentation method and system based on deep learning and attention mechanism | |
| Shi et al. | A method to detect earthquake-collapsed buildings from high-resolution satellite images | |
| Chandra et al. | Human cognition based framework for detecting roads from remote sensing images | |
| CN115035328A (en) | Converter image increment automatic machine learning system and establishment training method thereof | |
| Katrojwar et al. | Design of Image based Analysis and Classification using Unmanned Aerial Vehicle | |
| He et al. | MFI-YOLO: multi-fault insulator detection based on an improved YOLOv8 | |
| CN115661429B (en) | System and method for identifying defects of boiler water wall pipe and storage medium | |
| CN117152094A (en) | Method, device and system for analyzing surface defects of steel plate based on computer vision | |
| CN115690505A (en) | Photovoltaic module fault detection method and device, computer equipment and storage medium | |
| Devereux et al. | Automated object detection for visual inspection of nuclear reactor cores | |
| Hu et al. | An automatic detection method of the mural shedding disease using YOLOv4 | |
| Guo et al. | Automatic Detection and Dimensional Measurement of Minor Concrete Cracks with Convolutional Neural Network | |
| Hadjidemetriou et al. | Patch defects detection for pavement assessment, using smartphones and support vector machines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |