CN113378744A - Power transmission line inspection target identification method and device - Google Patents
Power transmission line inspection target identification method and device Download PDFInfo
- Publication number
- CN113378744A CN113378744A CN202110687737.5A CN202110687737A CN113378744A CN 113378744 A CN113378744 A CN 113378744A CN 202110687737 A CN202110687737 A CN 202110687737A CN 113378744 A CN113378744 A CN 113378744A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- transmission line
- module
- power transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 55
- 238000007689 inspection Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000011218 segmentation Effects 0.000 claims abstract description 48
- 230000007613 environmental effect Effects 0.000 claims abstract description 45
- 230000003416 augmentation Effects 0.000 claims abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims description 55
- 230000000694 effects Effects 0.000 claims description 29
- 230000003190 augmentative effect Effects 0.000 claims description 28
- 238000007781 pre-processing Methods 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 20
- 238000005520 cutting process Methods 0.000 claims description 18
- 230000004927 fusion Effects 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 10
- 238000007499 fusion processing Methods 0.000 claims description 5
- 238000005286 illumination Methods 0.000 abstract description 7
- 238000005070 sampling Methods 0.000 description 5
- 239000011800 void material Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 239000003595 mist Substances 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 241000533950 Leucojum Species 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000012212 insulator Substances 0.000 description 2
- 235000012736 patent blue V Nutrition 0.000 description 2
- 238000009826 distribution Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
- G07C1/20—Checking timed patrols, e.g. of watchman
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Economics (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method and a device for identifying a routing inspection target of a power transmission line, which are characterized in that a collected original image is preprocessed to extract a processed image containing an identification target, the processed image is deformed and added with environmental conditions, so that a deformed image and/or a processed image with combined environmental conditions is obtained to simulate images at different shooting angles and under different environmental conditions, and the processed image, the deformed image and the added image after augmentation operation jointly form an image data set, so that the data volume and diversity are improved, the image data set is input into a target segmentation network model trained by a segmentation network, the identification target in the power transmission line at different shooting angles and under different environments can be quickly and accurately identified, and the influence of the environments such as background, weather, illumination and the like on the extraction of the identification target in the power transmission line is reduced, the difficulty of identifying and positioning the power transmission line is reduced.
Description
Technical Field
The application relates to the technical field of power transmission line inspection, in particular to a method and a device for identifying inspection target marks of power transmission lines.
Background
Adopt unmanned aerial vehicle operation in fields such as transmission line patrols and examines, not only can guarantee operation personnel's personal safety, still will practice thrift a large amount of human costs, improve production efficiency.
However, in an outdoor unstructured operation environment, images acquired by the visual system of the unmanned aerial vehicle are susceptible to influences of various environmental factors such as background, weather and illumination, so that the extraction effect of the power transmission line is greatly interfered, and the difficulty in identifying and positioning the power transmission line is increased.
Therefore, a method for identifying a line inspection target capable of reducing interference of environmental factors is needed.
Disclosure of Invention
The application provides a method and a device for identifying a power transmission line inspection target, which are used for solving the technical problem that the difficulty of identifying and positioning the power transmission line is increased due to the fact that an acquired image is easily influenced by environmental factors.
In view of this, the first aspect of the present application provides a method for identifying a power transmission line inspection target, including the following steps:
preprocessing an original image of a pre-collected power transmission line to obtain a processed image, wherein the processed image contains an identification target;
carrying out deformation processing on the processed image to obtain a deformed image;
adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition;
respectively carrying out augmentation operation on the processed image, the deformed image and the additional image to obtain augmented image data sets;
training a segmentation network according to the augmented image data set to obtain a target segmentation network model for outputting a recognition target corresponding to the augmented image data set;
and extracting the recognition target in the original image to be recognized through the target segmentation network model.
Preferably, the preprocessing is performed on the pre-acquired original image of the power transmission line to obtain a processed image, and the step of including the identification target in the processed image specifically includes:
cutting the original image to obtain a plurality of cut images with consistent sizes;
automatically and/or manually labeling a cutting image containing the identification target in a plurality of cutting images, so that the labeled cutting images are used as processing images.
Preferably, the step of deforming the processed image to obtain a deformed image specifically includes:
and performing deformation processing on the processed image in one or more deformation modes of horizontal overturning, vertical overturning, random angle rotation, 90-degree rotation, 180-degree rotation, 270-degree rotation, random scaling and distortion deformation to obtain the deformed image.
Preferably, the step of adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with an environmental condition specifically includes:
and performing one or more of brightness operation, background operation and image adding operation of weather effect operation on the processed image and/or the deformed image according to environmental conditions to obtain the additional image with the environmental conditions, wherein the additional image comprises a brightness image, a background image and an effect image.
Preferably, the segmentation network includes an encoding layer and a decoding layer, and the step of training the segmentation network according to the augmented image data set to obtain a segmentation network model for outputting the recognition target corresponding to the augmented image data set specifically includes:
inputting the augmented image data set to the coding layer for coding processing to obtain a plurality of corresponding target images;
performing fusion processing on the plurality of target images to obtain target fusion images;
and inputting the target fusion image into the decoding layer for decoding processing, and obtaining a target image containing an identification target based on a decoding processing result.
Preferably, the coding layer comprises a hole convolution and a hole space convolution pooling pyramid.
In a second aspect, the present invention further provides a device for identifying a routing inspection target of a power transmission line, including:
the device comprises a preprocessing module, a processing module and a processing module, wherein the preprocessing module is used for preprocessing an original image of a pre-collected power transmission line to obtain a processed image, and the processed image contains an identification target;
the deformation module is used for carrying out deformation processing on the processed image to obtain a deformed image;
an image processing module, configured to add an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition;
the augmentation module is used for respectively performing augmentation operation on the processed image, the deformed image and the additional image to obtain an augmented image data set;
the training module is used for training a segmentation network according to the augmented image data set so as to obtain a target segmentation network model for outputting a recognition target corresponding to the augmented image data set;
and the extraction module is used for extracting the identification target in the original image to be identified through the target segmentation network model.
Preferably, the preprocessing module specifically includes:
the cutting submodule is used for cutting the original image to obtain a plurality of cutting images with consistent sizes;
and the labeling sub-module is used for automatically and/or manually labeling the cut images containing the identification targets in the plurality of cut images so as to take the labeled cut images as processing images.
Preferably, the deformation module is specifically configured to perform deformation processing on the processed image in one or a combination of a horizontal flip, a vertical flip, a random angle rotation, a 90 ° rotation, a 180 ° rotation, a 270 ° rotation, a random scaling and a distortion deformation to obtain the deformed image;
the image processing module is specifically configured to perform one or more of a brightness operation, a background operation, and an image adding operation of a weather effect operation on the processed image and/or the deformed image according to an environmental condition to obtain the additional image with the environmental condition, where the additional image includes a brightness image, a background image, and an effect image.
Preferably, the split network comprises an encoding module and a decoding module; the training module comprises a first input submodule, a fusion submodule and a second input submodule;
the first input submodule is used for inputting the amplified image data set to the coding module for coding processing to obtain a plurality of corresponding target images;
the fusion sub-module is used for carrying out fusion processing on the plurality of target images to obtain target fusion images;
and the second input submodule is used for inputting the target fusion image into the decoding module for decoding processing, and obtaining a target image containing an identification target based on a decoding processing result.
According to the technical scheme, the invention has the following advantages:
the invention provides a method and a device for identifying a power transmission line inspection target, which extracts a processing image containing an identification target by preprocessing an acquired original image, deforms and attaches an environmental condition to the processing image to obtain a deformed image and/or a processing image combined with the environmental condition so as to simulate images at different shooting angles and under different environmental conditions, and an image data set is formed by the processing image, the deformed image and the additional image after augmentation operation together, thereby improving the data volume and diversity, inputting the image data set into a target segmentation network model trained by a segmentation network, being capable of rapidly and accurately identifying the identification target in the power transmission line at different shooting angles and under different environments, thereby reducing the influence of the environments such as background, weather, illumination and the like on the extraction of the identification target in the power transmission line, the difficulty of identifying and positioning the power transmission line is reduced.
Drawings
Fig. 1 is a flowchart of a method for identifying a power transmission line inspection target according to an embodiment of the present application;
fig. 2 is a structural diagram of a split network according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a transmission line inspection target identification device provided in the embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived by a person skilled in the art from the embodiments given in the present application without making any creative effort shall fall within the protection scope of the present application.
For convenience of understanding, please refer to fig. 1, the method for identifying the routing inspection target of the power transmission line provided by the present application includes the following steps:
s1, preprocessing the pre-collected original image of the power transmission line to obtain a processed image, wherein the processed image contains an identification target;
the method includes the steps of acquiring an original image through a camera device such as a video camera, a mobile phone, a digital camera and the like, and shooting a power transmission line through the camera device arranged on a mobile device to acquire the original image, wherein the mobile device is a mobile robot or a flying robot, the power transmission line is composed of devices such as a line tower, a wire, an insulator, a line fitting, a stay wire, a tower foundation, a grounding device and the like, and the devices on the power transmission line can be used as identification targets. Meanwhile, the original image comprises one or more recognition targets.
S2, carrying out deformation processing on the processed image to obtain a deformed image;
s3, adding environmental conditions to the processed image and/or the deformed image to obtain an additional image with the environmental conditions;
s4, performing augmentation operation on the processed image, the deformed image and the additional image respectively to obtain augmented image data sets;
it can be understood that the processing image, the deformed image and the additional image are respectively subjected to the augmentation operation, so that the data volume can be improved, and the accuracy of the training result can be further improved.
S5, training the segmentation network according to the augmented image data set to obtain a target segmentation network model for outputting a recognition target corresponding to the augmented image data set;
and S6, extracting the recognition target in the original image to be recognized through the target segmentation network model.
The embodiment extracts the processing image containing the recognition target by preprocessing the acquired original image, and deforms and attaches the environmental conditions to the processing image, so as to obtain a deformed image and/or a processed image combined with the environmental conditions to simulate the images at different shooting angles and under different environmental conditions, the processed image, the deformed image and the additional image after the augmentation operation form an image data set together, so that the data volume and the diversity are improved, the image data set is input into a target segmentation network model trained by a segmentation network, the recognition targets in the power transmission line under different shooting angles and different environments can be quickly and accurately recognized, therefore, the influence of the environment such as background, weather and illumination on the extraction of the identification target in the power transmission line is reduced, and the difficulty in identifying and positioning the power transmission line is reduced.
Further, step S1 specifically includes:
s101, cutting the original image to obtain a plurality of cut images with consistent sizes;
the size of the cutting image can be 1/10-1/5 of the original image. In a general example, the original image can be seamlessly cut using the gdal module in python software.
And S102, automatically and/or manually labeling the cut images containing the recognition targets in the plurality of cut images, so that the labeled cut images are used as processing images.
It can be understood that labeling can be completed by manually labeling the cut image by using labelme software or by recognizing the cut image based on a target recognition model trained by a depth learning algorithm.
Further, step S2 specifically includes:
and deforming the processed image in one or more deformation modes of horizontal overturning, vertical overturning, random angle rotation, 90-degree rotation, 180-degree rotation, 270-degree rotation, random scaling and distortion deformation to obtain a deformed image.
It can be understood that, through the above-mentioned various deformation modes, scenes of acquiring images at different shooting angles can be simulated, so as to improve data diversity.
Further, the step of adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition specifically includes:
and performing one or more of brightness operation, background operation and image adding operation of weather effect operation on the processed image and/or the deformed image according to the environmental condition to obtain an additional image with the environmental condition, wherein the additional image comprises the brightness image, the background image and the effect image.
Specifically, the brightness operation is performed on the processed image and/or the deformed image to obtain a brightness image, that is, the processed images at different brightness are acquired to simulate the images acquired under different illumination.
And deforming the background color of the processed image and/or the deformed image to obtain a background image. If the background color of the processed image is generally sky blue in sunny weather, and the background color of the processed image is red when the processed image is shot at the time with sunset in the evening, the background color of the processed image can be adjusted to simulate the images acquired in the background of different sky.
And adding weather effect to the processed image and/or the deformed image to obtain an effect image. For example, adding rain effects to the processed image simulates the image captured during rainy weather. And adding a mist effect in the processed image to simulate the acquired image in the mist or haze weather. And adding a rainwater effect into the processed image, and adding a snowflake effect into the processed image so as to simulate the acquired image in snowing weather.
By carrying out the image operation on the deformed image, the image acquired under a multi-deformation multi-environment scene can be simulated. Therefore, the data volume is improved, and the accuracy of the training result is improved.
Further, the structure of the split network is shown in fig. 2, and the split network includes an encoding layer and a decoding layer; step S5 specifically includes:
s501, inputting the augmented image data set to a coding layer for coding processing to obtain a plurality of corresponding target images;
s502, fusing the target images to obtain target fused images;
and S503, inputting the target fusion image into a decoding layer for decoding, and obtaining a target image containing the recognition target based on the decoding result.
Further, the coding layer includes a hole convolution and a hole space convolution pooling pyramid.
Specifically, compared with the conventional convolution kernel, the hole convolution has one more superparameter, namely, the hole rate, and represents the number of intervals between convolution kernels (the interval between normal convolutions is considered to be 1). The empty hole convolution increases the receptive field on the premise of not performing down-sampling operation (such as pooling) and enables each convolution output to contain information in a larger range, thereby reducing the loss of spatial position information caused by down-sampling. Specifically, the receptive field calculation formula of the convolution of a single hole is as follows:
n＝k+(k-1)×(d-1)
where k is the size of the convolution kernel, d is the void rate, and n is the receptive field of a single convolution kernel.
The receptive field calculation formula of each characteristic layer of the hole convolution is as follows:
wherein lm-1The size of the receptive field corresponding to the m-1 st layer, fmThe pooled nuclear size of the m-th layer, siThe pooled kernel step size for the ith layer.
The cavity space convolution pooling pyramid is known by a single cavity convolution receptive field calculation formula and a cavity convolution receptive field calculation formula of each characteristic layer, characteristic graphs obtained by different cavity rates correspond to different receptive fields and context information of different scales, and a network is helped to capture multi-scale information through combination of the cavity convolution layers of different cavity rates. By constructing multi-scale context information, the segmentation effect on different-scale targets is improved.
The void space convolution pooling pyramid comprises a convolution kernel and three void convolution kernels, wherein the convolution layer is a 1 multiplied by 1 convolution layer; the void volume layers were 3X 3 volume layers, and the void ratios were 6, 12, and 18, respectively.
The convolution kernel and the hole convolution kernel are 256 channels and a batch normalization operation is introduced. And obtaining a characteristic diagram through global average pooling, convolution kernel and bilinear interpolation. The feature maps obtained above were spliced and then convolved using 256 1 × 1 convolution layers to obtain a fused feature map. And finally, outputting the fused feature map by the coding layer.
Inputting the augmented image data set into a cavity convolution, inputting a convolution image output by the cavity convolution into a cavity space convolution pooling pyramid, outputting a fused feature map by the cavity space convolution pooling pyramid, splicing the convolution image with the corresponding fused feature map subjected to quadruple up-sampling after passing through 48 1 × 1 convolution layers, and obtaining a first decoding splicing map. The first decoding mosaic uses 256 1 × 1 convolution layers to perform channel fusion, performs double up sampling once, and then passes through another 48 1 × 1 convolution layers with the convolution image to perform mosaic to obtain a second decoding mosaic. The second decoding splicing diagram uses a 3 multiplied by 3 convolutional layer to carry out convolution operation, and finally carries out double up-sampling to calculate a loss output result.
The segmentation network integrates the hole convolution, the hole convolution is used for increasing the receptive field, each convolution output contains information in a large range, and the output feature graph is denser; capturing multi-scale context information, and improving the segmentation effect on different scale recognition targets; and shallow information is fused, and the accurate position segmentation capability of the recognition target is enhanced.
The information is obvious to some characteristics, such as color, shape and other characteristics, and the characteristics are often present in a shallow network. Compared with high-level features, the shallow-level feature map has smaller receptive field and higher accuracy of extracting the position information. The segmentation network of the invention enhances the shallow feature to obtain more accurate edge segmentation effect.
The loss function employed in a split network is typically an average weighted cross entropy loss. Different categories have different weight coefficients, and less categories adopt large weight coefficients, so that the problem of unbalanced category distribution can be solved. The loss function CE can be expressed as:
where N is the total number of pixels, i is the current pixel, j is the current category, C is the total number of categories, wjIs a weight coefficient of the jth class,
is the predicted value of the current pixel point i on the j-th class,
is the label value of the current pixel point i on the jth class.
Different weighting coefficients are selected to have different influences on the segmentation effect of the segmented network. The weighting coefficients are set to 1,2,3, 4,5, respectively. 40000 times of training the segmentation network by using the augmented image set to obtain a segmentation model, and the evaluation indexes, the intersection ratio, the accuracy and the recall rate of the segmentation model are shown in the following table 1.
Table 1: result of identifying target under different weight coefficients under target segmentation model
| Weight coefficient | Cross ratio of | Rate of accuracy | Recall rate |
| 1 | 0.9733 | 0.9818 | 0.9942 |
| 2 | 0.9715 | 0.9798 | 0.9914 |
| 3 | 0.9703 | 0.9779 | 0.9921 |
| 4 | 0.9637 | 0.9664 | 0.9971 |
| 5 | 0.9606 | 0.9644 | 0.9959 |
When the weighting coefficients are respectively set to 1, the intersection ratio of the whole data set is the maximum, which is 0.9733, and the accuracy is the maximum at this time, which reaches 0.9818. The intersection-specific difference of the weight coefficients at 1,2 and 3 is not large as a whole. But when the weight reaches 4,5, the accuracy rate and the intersection ratio are obviously reduced. To ensure a high intersection ratio and accuracy and a suitable recall ratio, the weighting factor is preferably 1.
Setting the weight coefficient of the segmentation network to be 1, repeating training on the segmentation network through the augmented image data set, preferably training 10000-50000 times, preferably 40000 times, so as to obtain a target segmentation network model, and extracting the recognition target in the original image to be recognized by the segmentation model.
The above is a detailed description of an embodiment of the method for identifying the transmission line inspection target provided by the invention, and the following is a detailed description of the device for identifying the transmission line inspection target provided by the invention.
For convenience of understanding, please refer to fig. 3, the present invention further provides a device for identifying a target for routing inspection of a power transmission line, comprising:
the system comprises a preprocessing module 100, a recognition module and a processing module, wherein the preprocessing module 100 is used for preprocessing an original image of a pre-collected power transmission line to obtain a processed image, and the processed image contains a recognition target;
the method includes the steps of acquiring an original image through a camera device such as a video camera, a mobile phone, a digital camera and the like, and shooting a power transmission line through the camera device arranged on a mobile device to acquire the original image, wherein the mobile device is a mobile robot or a flying robot, the power transmission line is composed of devices such as a line tower, a wire, an insulator, a line fitting, a stay wire, a tower foundation, a grounding device and the like, and the devices on the power transmission line can be used as identification targets. Meanwhile, the original image comprises one or more recognition targets.
A deformation module 200, configured to perform deformation processing on the processed image to obtain a deformed image;
an image processing module 300 for adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition;
an augmentation module 400, configured to perform augmentation operations on the processed image, the deformed image, and the additional image, respectively, to obtain augmented image datasets;
it can be understood that the processing image, the deformed image and the additional image are respectively subjected to the augmentation operation, so that the data volume can be improved, and the accuracy of the training result can be further improved.
A training module 500, configured to train a segmentation network according to the augmented image data set to obtain a target segmentation network model that outputs a recognition target corresponding to the augmented image data set;
the extracting module 600 is configured to extract an identification target in an original image to be identified through the target segmentation network model.
The embodiment extracts the processing image containing the recognition target by preprocessing the acquired original image, and deforms and attaches the environmental conditions to the processing image, so as to obtain a deformed image and/or a processed image combined with the environmental conditions to simulate the images at different shooting angles and under different environmental conditions, the processed image, the deformed image and the additional image after the augmentation operation form an image data set together, so that the data volume and the diversity are improved, the image data set is input into a target segmentation network model trained by a segmentation network, the recognition targets in the power transmission line under different shooting angles and different environments can be quickly and accurately recognized, therefore, the influence of the environment such as background, weather and illumination on the extraction of the identification target in the power transmission line is reduced, and the difficulty in identifying and positioning the power transmission line is reduced.
Further, the preprocessing module specifically includes:
the cutting submodule is used for cutting the original image to obtain a plurality of cutting images with consistent sizes;
the size of the cutting image can be 1/10-1/5 of the original image. In a general example, the cutting sub-module may use the gdal module in python software to make a seamless cut of the original image.
And the labeling sub-module is used for automatically and/or manually labeling the cut images containing the recognition targets in the plurality of cut images so as to take the labeled cut images as processing images.
It can be understood that labeling can be completed by manually labeling the cut image by using labelme software or by recognizing the cut image based on a target recognition model trained by a depth learning algorithm.
Further, the deformation module is specifically configured to perform deformation processing on the processed image in one or a combination of multiple deformation modes of horizontal flipping, vertical flipping, random angle rotation, 90 ° rotation, 180 ° rotation, 270 ° rotation, random scaling and distortion deformation to obtain a deformed image;
it can be understood that, through the above-mentioned various deformation modes, scenes of acquiring images at different shooting angles can be simulated, so as to improve data diversity.
The image processing module is specifically configured to perform one or more of a brightness operation, a background operation, and an image adding operation of a weather effect operation on the processed image and/or the deformed image according to the environmental condition to obtain an additional image with the environmental condition, where the additional image includes a brightness image, a background image, and an effect image.
Specifically, the brightness operation is performed on the processed image and/or the deformed image to obtain a brightness image, that is, the processed images at different brightness are acquired to simulate the images acquired under different illumination.
And deforming the background color of the processed image and/or the deformed image to obtain a background image. If the background color of the processed image is generally sky blue in sunny weather, and the background color of the processed image is red when the processed image is shot at the time with sunset in the evening, the background color of the processed image can be adjusted to simulate the images acquired in the background of different sky.
And adding weather effect to the processed image and/or the deformed image to obtain an effect image. For example, adding rain effects to the processed image simulates the image captured during rainy weather. And adding a mist effect in the processed image to simulate the acquired image in the mist or haze weather. And adding a rainwater effect into the processed image, and adding a snowflake effect into the processed image so as to simulate the acquired image in snowing weather.
By carrying out the image operation on the deformed image, the image acquired under a multi-deformation multi-environment scene can be simulated. Therefore, the data volume is improved, and the accuracy of the training result is improved.
Further, the split network comprises an encoding module and a decoding module; the training module comprises a first input submodule, a fusion submodule and a second input submodule;
the first input submodule is used for inputting the amplified image data set to the coding module for coding processing to obtain a plurality of corresponding target images;
the fusion submodule is used for carrying out fusion processing on the plurality of target images to obtain a target fusion image;
the second input submodule is used for inputting the target fusion image into the decoding module for decoding processing, and obtaining a target image containing the recognition target based on the result of the decoding processing.
The coding layer comprises a cavity convolution and a cavity space convolution pooling pyramid.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a hardware form, and can also be realized in a software functional unit form.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A transmission line inspection target identification method is characterized by comprising the following steps:
preprocessing an original image of a pre-collected power transmission line to obtain a processed image, wherein the processed image contains an identification target;
carrying out deformation processing on the processed image to obtain a deformed image;
adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition;
respectively carrying out augmentation operation on the processed image, the deformed image and the additional image to obtain augmented image data sets;
training a segmentation network according to the augmented image data set to obtain a target segmentation network model for outputting a recognition target corresponding to the augmented image data set;
and extracting the recognition target in the original image to be recognized through the target segmentation network model.
2. The method for identifying the inspection target of the power transmission line according to claim 1, wherein the preprocessing is performed on an original image of the power transmission line acquired in advance to obtain a processed image, and the step of including the identification target in the processed image specifically comprises the following steps:
cutting the original image to obtain a plurality of cut images with consistent sizes;
automatically and/or manually labeling the cut images containing the recognition targets in a plurality of the cut images, thereby using the labeled cut images as processing images.
3. The method for identifying the power transmission line inspection target according to claim 1, wherein the step of deforming the processed image to obtain a deformed image specifically comprises:
and performing deformation processing on the processed image in one or more deformation modes of horizontal overturning, vertical overturning, random angle rotation, 90-degree rotation, 180-degree rotation, 270-degree rotation, random scaling and distortion deformation to obtain the deformed image.
4. The method for identifying the power transmission line inspection target according to claim 1, wherein the step of adding an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition specifically comprises:
and performing one or more of brightness operation, background operation and image adding operation of weather effect operation on the processed image and/or the deformed image according to the environmental condition to obtain the additional image with the environmental condition, wherein the additional image comprises a brightness image, a background image and an effect image.
5. The method for identifying the inspection target of the power transmission line according to claim 1, wherein the segmentation network comprises an encoding layer and a decoding layer, and the step of training the segmentation network according to the augmented image data set to obtain the segmentation network model for outputting the identification target corresponding to the augmented image data set specifically comprises:
inputting the augmented image data set to the coding layer for coding processing to obtain a plurality of corresponding target images;
performing fusion processing on the plurality of target images to obtain target fusion images;
and inputting the target fusion image into the decoding layer for decoding processing, and obtaining a target image containing an identification target based on a decoding processing result.
6. The method for identifying the power transmission line inspection target according to claim 5, wherein the coding layer comprises a hole convolution and a hole space convolution pooling pyramid.
7. The utility model provides a transmission line patrols and examines target identification device which characterized in that includes:
the device comprises a preprocessing module, a processing module and a processing module, wherein the preprocessing module is used for preprocessing an original image of a pre-collected power transmission line to obtain a processed image, and the processed image contains an identification target;
the deformation module is used for carrying out deformation processing on the processed image to obtain a deformed image;
an image processing module, configured to add an environmental condition to the processed image and/or the deformed image to obtain an additional image with the environmental condition;
the augmentation module is used for respectively carrying out augmentation operation on the processed image, the deformed image and the additional image to obtain an augmented image data set;
the training module is used for training a segmentation network according to the augmented image data set so as to obtain a target segmentation network model for outputting a recognition target corresponding to the augmented image data set;
and the extraction module is used for extracting the identification target in the original image to be identified through the target segmentation network model.
8. The device for identifying the power transmission line inspection target according to claim 7, wherein the preprocessing module specifically comprises:
the cutting submodule is used for cutting the original image to obtain a plurality of cutting images with consistent sizes;
and the labeling sub-module is used for automatically and/or manually labeling the cut images containing the identification targets in the plurality of cut images so as to take the labeled cut images as processing images.
9. The device for identifying the inspection target of the power transmission line according to claim 7, wherein the deformation module is specifically configured to perform deformation processing on the processed image in one or a combination of a plurality of deformation modes selected from horizontal inversion, vertical inversion, random angular rotation, 90 ° rotation, 180 ° rotation, 270 ° rotation, random scaling and distortion deformation to obtain the deformed image;
the image processing module is specifically configured to perform one or more of a brightness operation, a background operation, and an image adding operation of a weather effect operation on the processed image and/or the deformed image according to an environmental condition to obtain the additional image with the environmental condition, where the additional image includes a brightness image, a background image, and an effect image.
10. The power transmission line inspection target identification device according to claim 7, wherein the segmentation network includes an encoding module and a decoding module; the training module comprises a first input submodule, a fusion submodule and a second input submodule;
the first input submodule is used for inputting the amplified image data set to the coding module for coding processing to obtain a plurality of corresponding target images;
the fusion sub-module is used for carrying out fusion processing on the plurality of target images to obtain target fusion images;
and the second input submodule is used for inputting the target fusion image into the decoding module for decoding processing, and obtaining a target image containing an identification target based on a decoding processing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110687737.5A CN113378744A (en) | 2021-06-21 | 2021-06-21 | Power transmission line inspection target identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110687737.5A CN113378744A (en) | 2021-06-21 | 2021-06-21 | Power transmission line inspection target identification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378744A true CN113378744A (en) | 2021-09-10 |
Family
ID=77578173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110687737.5A Pending CN113378744A (en) | 2021-06-21 | 2021-06-21 | Power transmission line inspection target identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378744A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114529561A (en) * | 2022-02-22 | 2022-05-24 | 成都数之联科技股份有限公司 | Method, system, device and medium for segmenting line image and background plate image |
| CN114998758A (en) * | 2022-05-26 | 2022-09-02 | 电子科技大学 | Power transmission line insulator detection method based on multi-source remote sensing satellite image |
| CN116503396A (en) * | 2023-06-26 | 2023-07-28 | 广东电网有限责任公司湛江供电局 | Insulator pollution flashover rapid detection method, system and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344883A (en) * | 2018-09-13 | 2019-02-15 | 西京学院 | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution |
| US20200349364A1 (en) * | 2019-04-30 | 2020-11-05 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for training lane line identifying model, device, and storage medium |
| CN111967393A (en) * | 2020-08-18 | 2020-11-20 | 杭州师范大学 | Helmet wearing detection method based on improved YOLOv4 |
-
2021
- 2021-06-21 CN CN202110687737.5A patent/CN113378744A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344883A (en) * | 2018-09-13 | 2019-02-15 | 西京学院 | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution |
| US20200349364A1 (en) * | 2019-04-30 | 2020-11-05 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for training lane line identifying model, device, and storage medium |
| CN111967393A (en) * | 2020-08-18 | 2020-11-20 | 杭州师范大学 | Helmet wearing detection method based on improved YOLOv4 |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114529561A (en) * | 2022-02-22 | 2022-05-24 | 成都数之联科技股份有限公司 | Method, system, device and medium for segmenting line image and background plate image |
| CN114998758A (en) * | 2022-05-26 | 2022-09-02 | 电子科技大学 | Power transmission line insulator detection method based on multi-source remote sensing satellite image |
| CN114998758B (en) * | 2022-05-26 | 2024-05-03 | 电子科技大学 | Transmission line insulator detection method based on multisource remote sensing satellite images |
| CN116503396A (en) * | 2023-06-26 | 2023-07-28 | 广东电网有限责任公司湛江供电局 | Insulator pollution flashover rapid detection method, system and equipment |
| CN116503396B (en) * | 2023-06-26 | 2024-02-27 | 广东电网有限责任公司湛江供电局 | Insulator pollution flashover rapid detection method, system and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113378744A (en) | Power transmission line inspection target identification method and device | |
| CN109871798A (en) | A kind of remote sensing image building extracting method based on convolutional neural networks | |
| CN108629368B (en) | Multi-modal foundation cloud classification method based on joint depth fusion | |
| CN111814771B (en) | Image processing method and device | |
| CN113408594B (en) | Remote sensing scene classification method based on attention network scale feature fusion | |
| CN110097106A (en) | The low-light-level imaging algorithm and device of U-net network based on deep learning | |
| CN116052016A (en) | Fine segmentation detection method for remote sensing image cloud and cloud shadow based on deep learning | |
| CN113887472A (en) | Remote sensing image cloud detection method based on cascade color and texture feature attention | |
| CN103605171A (en) | All-sky imaging instrument and cloud layer characteristic analysis method based on all-sky imaging instrument | |
| CN111274988B (en) | Multispectral-based vehicle weight identification method and device | |
| CN115984672B (en) | Detection method and device for small target in high-definition image based on deep learning | |
| CN113936019A (en) | Method for estimating field crop yield based on convolutional neural network technology | |
| CN117574317A (en) | Mountain fire monitoring method and device based on multi-mode data fusion of sky and ground | |
| CN113378672A (en) | Multi-target detection method for defects of power transmission line based on improved YOLOv3 | |
| CN107194340B (en) | Method and system for searching mountain-climbing distress people by using computer vision | |
| CN111898671B (en) | Target identification method and system based on fusion of laser imager and color camera codes | |
| CN113361355B (en) | Remote sensing image earth surface element identification method combining multi-temporal characteristics and storage medium | |
| CN111402223B (en) | Transformer substation defect problem detection method using transformer substation video image | |
| CN107895385A (en) | The Forecasting Methodology to individual outdoor images position of sun based on convolutional neural networks | |
| CN115115713A (en) | Unified space-time fusion all-around aerial view perception method | |
| CN112633158A (en) | Power transmission line corridor vehicle identification method, device, equipment and storage medium | |
| CN113887124A (en) | Method for predicting photovoltaic power station cloud layer distribution state under multi-dimensional mixed weather | |
| CN115719368B (en) | Multi-target ship tracking method and system | |
| CN114359541A (en) | Fault insulator detection method based on candidate frame generation and attention coordination mechanism | |
| CN116958834A (en) | Large-scale greenhouse interpretation method based on unmanned aerial vehicle image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |