CN117809178A - Road abnormity warning method, system and medium in power transmission channel scene - Google Patents
Road abnormity warning method, system and medium in power transmission channel scene Download PDFInfo
- Publication number
- CN117809178A CN117809178A CN202311709006.1A CN202311709006A CN117809178A CN 117809178 A CN117809178 A CN 117809178A CN 202311709006 A CN202311709006 A CN 202311709006A CN 117809178 A CN117809178 A CN 117809178A
- Authority
- CN
- China
- Prior art keywords
- road
- power transmission
- transmission channel
- attention
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 107
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000011218 segmentation Effects 0.000 claims abstract description 73
- 238000001514 detection method Methods 0.000 claims abstract description 31
- 230000005856 abnormality Effects 0.000 claims abstract description 26
- 238000012544 monitoring process Methods 0.000 claims abstract description 15
- 238000002372 labelling Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims description 35
- 238000011176 pooling Methods 0.000 claims description 26
- 230000007246 mechanism Effects 0.000 claims description 17
- 230000004913 activation Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000002159 abnormal effect Effects 0.000 claims 4
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 238000010606 normalization Methods 0.000 description 6
- 239000004568 cement Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000012806 monitoring device Methods 0.000 description 3
- 241000282326 Felis catus Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000010792 warming Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention relates to a road abnormality warning method, a system and a medium in a power transmission channel scene, and relates to the technical field of computer vision. The method comprises the following steps of S1: and acquiring a power transmission channel scene image of the road, and using an X-AnyLabeling tool to semi-automatically carry out pixel-level polygonal labeling on a road region in the power transmission channel scene image by using the SAM model so as to construct a road segmentation data set in the power transmission channel scene. S2: and carrying out segmentation training on the improved PPLiteSeg model by using the road segmentation data set, and finally obtaining the power transmission channel road segmentation model. S3: the road segmentation model of the trained road segmentation transmission channel is deployed into monitoring equipment of the transmission line, the road region is accurately segmented by using the road segmentation model, and the segmentation result is linked with the detection result of the detection model to accurately divide the alarm level.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a road abnormality warning method, a system and a medium in a power transmission channel scene.
Background
The detection technology on the power transmission channel is gradually approaching to maturity, and the detection performance of the algorithm model is also increasingly accurate and rapid. However, with the increase of thousands of visual monitoring devices around the country, hundreds of millions of images are uploaded and analyzed every day, and tens of millions of alarming images are generated after model analysis, so that the workload of operation and inspection of related inspection staff is increased. With the development of the age, the progress of technology and how to lighten the workload of the related patrol personnel for transportation and inspection are required. Therefore, the invalid alarms are required to be filtered as much as possible, and only the valid alarms are left for relevant patrols to check, so that the analysis load of the patrols is reduced.
The effective alarm refers to external broken machinery, smoke, fire, wire foreign matters and the like which are under construction under the transmission channel line, and the moving trend of the equipment is about to threat the line. The invalid warning means that the small-sized machinery which is far away from the line channel and does not cause harm to the transmission overhead line tower, the hidden danger which is formed, or the non-working crane, the non-working cement pump truck, the cement mixer truck and the like which do not run on the road, keep the safe distance with the line, means that the hidden danger does not move to the lower side of the lead and possibly does not enter the lower side of the lead.
In order to filter out part of invalid alarms, such as a non-working crane, a non-working cement pump truck, a cement mixer truck and the like which do not run on a road, the road area is firstly required to be effectively identified, the intersection ratio of the road area and the hidden danger identified by the hidden danger detection model of the power transmission channel is calculated, and whether to alarm is determined by judging the relation between the intersection ratio of a hidden danger target and the road area and a preset threshold value.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the invention provides a road abnormality warning method, a system and a medium in a power transmission channel scene.
In a first aspect, the present invention provides a road abnormality warning method in a power transmission channel scene, including:
s1: acquiring a power transmission channel scene image of a road, using an X-AnyLabeling tool, and carrying out pixel-level polygonal labeling on a road region in the power transmission channel scene image by using a SAM model in a semi-automatic manner to construct a road segmentation data set in the power transmission channel scene;
s2: carrying out segmentation training on the improved PPLiteSeg model by using a road segmentation data set to finally obtain a transmission channel road segmentation model;
s3: the trained transmission channel road segmentation model is deployed into monitoring equipment of a transmission line, a road area where the monitoring equipment collects images is segmented through the transmission channel road segmentation model, and the segmentation result is linked with a detection result of a detection model for detecting abnormality to accurately divide the alarm level.
Further, in the step S1, performing semi-automatic labeling on the scene image of the power transmission channel road, and constructing the road segmentation data set includes:
acquiring a power transmission channel scene image containing a road area under a power transmission channel scene through visual monitoring equipment which is already installed on a power transmission line;
performing pixel-level polygonal labeling on the road area by using an X-AnyLabeling tool and a SAM model in a semi-automatic manner, wherein the label is read, and obtaining a json format file with labeled completion;
the marked files are processed according to the following steps of 7:1: and 2, dividing the ratio into a training set, a verification set and a test set to obtain a road segmentation data set.
Still further, the transmission path road segmentation model uses an STDC network to extract backbones of road features under transmission scenes as an entire transmission path road segmentation model, the STDC network of the encoder of the modified PPLiteSeg model includes ConvX1 layer, convX2 layer, stage3 layer, stage4 layer, stage5 layer, convX6 layer, and GlobalPool layer, where ConvX (x=1, 2, 6) contains a convolutional (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combination operation; stage3, stage4 and Stage5 are all STDC modules forming an STDC network, and the GlobalPool layer is a global average pooling layer.
Further, the structure of the Stage3 layer, the Stage4 layer and the Stage5 layer comprises two STDC modules with a stride of 1 and a stride of 2, wherein each of the STDC modules with a stride of 1 and a stride of 2 comprises a block1, a block2, a block3 and a block4 which are sequentially connected in series, and outputs of the block1, the block2, the block3 and the block4 are sent to fusion; block1 is ConvX with convolution kernel of 1×1, block2-4 is ConvX with convolution kernel of 3×3, convX contains convolution (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combination operations; the number of the channels of the blocks 1-3 is 1/2 pi, i is the serial number of the block, and the channel of the block4 is consistent with the channel of the block 3; for the STDC module with the stride of 2, the convolution kernel stride of block2 is 2, and the output of block1 is subjected to averaging pooling with the stride of 2 and then transmitted to fusion.
Still further, a multi-attention fusion module is used in the decoder to apply channel and spatial attention to enrich the fusion feature representation; the multi-attention fusion module first uses bilinear interpolation to input F high Upsampling to F with input low Equal size, and F high The up-sampled feature is denoted as F up The multi-attention fusion module is F up And F low As input, a mixed attention mechanism integrating a channel attention module and a space attention module sequentially is used for calculating attention weights; will F up And attention weight, F low And (1-attention weight) performing element-by-element multiplication operation to realize attention weighting, and performing element-by-element addition operation on the attention weighted features to obtain output.
Furthermore, after the channel attention module of the mixed attention mechanism carries out maximum pooling and average pooling on the input respectively, the maximum pooling and average pooling results are input into the MLP respectively, and two features processed by the MLP are activated through a sigmod function after being added.
Still further, the spatial attention module of the mixed attention mechanism averages and maximizes the input from the channel dimension, combines the averaged and maximized results, and then activates by a convolution and sigmod function.
Still further, the step S3 includes: and (5) dividing alarm grades according to the segmentation result and the IoU of the detection result of the detection model, wherein the alarm grades are classified into serious alarms, medium alarms and no alarms.
In a second aspect, the present invention provides a road abnormality warning system in a power transmission channel scene, to implement the road abnormality warning method in the power transmission channel scene, including:
the image acquisition module is used for acquiring scene images of the power transmission channel in real time or at fixed time through the monitoring camera;
the power transmission channel road segmentation module is used for identifying and segmenting the roads in the power transmission channel scene image based on the power transmission channel road segmentation model;
the detection module is used for detecting the abnormality of the scene image of the power transmission channel;
and the alarm module is used for accurately dividing the alarm level by linking the segmentation result with the detection result of the detection model for detecting the abnormality.
In a third aspect, the present invention provides a computer readable storage medium storing a computer program, where the computer program when executed by a processor implements the road abnormality warning method in the power transmission channel scene.
Compared with the prior art, the technical scheme provided by the embodiment of the invention has the following advantages:
the road abnormality warning method provided by the invention is characterized in that the PPLiteSeg segmentation model with more outstanding speed and accuracy is taken as a base line, and the PPLiteSeg segmentation model is improved and optimized so that the PPLiteSeg segmentation model is more suitable for segmenting a road in a power transmission scene. For the improved PPLiteSeg, the STDC network is used as a whole transmission channel road segmentation model to extract the backbond of the road characteristics under the transmission scene, fewer parameters are used to obtain the multiscale receptive field and multiscale information, meanwhile, the calculation complexity is reduced, and the speed of the network for extracting the characteristics is increased. The multi-attention fusion module is used in the decoder, and the multi-attention fusion module enriches fusion characteristic representations by applying channels and spatial attention, so that the accuracy of road region segmentation is improved.
The invention focuses on an automatic labeling strategy, takes an X-AnyLabeling tool as a basis, utilizes an open-source SAM model to carry out semi-automatic polygonal labeling on a pixel level on a road area, can be used only by manually finely adjusting an area with unreasonable segmentation, accelerates the speed of constructing a data set and reduces the labor cost.
According to the invention, on the application level, the results of the segmentation model and the detection model are processed at the same time, and the hidden danger in the current scene is classified into alarm levels by calculating the position relation between the results of the segmentation model and the detection model, so that different processing methods are generated.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flowchart of a road abnormality warning method in a power transmission channel scene according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a road label diagram in a power transmission channel scene provided by an embodiment of the present invention;
fig. 3 is a block diagram of a transmission channel road segmentation model formed by an improved PPLiteSeg model according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an STDC module according to an embodiment of the present invention;
fig. 5 is a table of architecture of an STDC network according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a multi-attention fusion module according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a road area segmentation effect according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a road abnormality warning system in a power transmission channel scene according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Example 1
Referring to fig. 1, an embodiment of the present invention provides a road abnormality warning method in a power transmission channel scene, including:
s1: and acquiring a power transmission channel scene image of a road through a power transmission line visual monitoring device, and using an X-AnyLabeling tool to semi-automatically carry out pixel-level polygon labeling on a road region in the power transmission channel scene image data by using a SAM model so as to construct a road segmentation data set in the power transmission channel scene. The method comprises the following specific steps:
s11: and acquiring an image containing the road area in the scene of the power transmission channel through a visual monitoring device which is already installed on the power transmission line. S12: and carrying out pixel-level polygonal labeling on the road region in the scene image data of the power transmission channel by utilizing an X-AnyLabeling tool and utilizing a SAM model in a semi-automatic mode, wherein the label is read, and obtaining a json format file with a labeled picture as shown in figure 2. S13: the marked files are processed according to the following steps of 7:1: and 2, dividing the ratio into a training set, a verification set and a test set to obtain a road segmentation data set.
S2: and constructing a power transmission channel road segmentation model. And training the improved transmission channel road segmentation model by using the constructed road segmentation data set.
The PPLiteSeg model is taken as a base line on an algorithm model, and is improved to be more suitable for dividing the road in the power transmission scene, and finally the power transmission channel road division model is obtained.
The improved PPLiteSeg adopts a codec architecture, as shown in fig. 3, the STDC network is used as a whole transmission channel road segmentation model to extract the backbond of the road characteristics under the transmission scene, so that fewer parameters are used to obtain the multiscale receptive field and multiscale information, the calculation complexity is reduced, and the speed of the network for extracting the characteristics is increased.
Specifically, the network architecture of the STDC network of the encoder is shown in fig. 5, and includes ConvX1 layer, convX2 layer, stage3 layer, stage4 layer, stage5 layer, convX6 layer, and GlobalPool layer, where ConvX (x=1, 2, 6) contains a convolutional (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combining operation. Stage3, stage4, and Stage5 are all STDC modules forming an STDC network. KSize denotes the size of the convolution kernel, and S, R, C denotes the stride, the number of repetitions, and the number of output channels, respectively.
The ConvX1 at stage 1 and the ConvX2 at stage 2 both use only one convolution layer with 3×3 convolution kernel and 2 stride, and the stride s=2, so after the ConvX1 and ConvX2 processing, the resolution of the feature map decreases by half, and after the 2 nd stage is finished, the feature map resolution= (H/4, w/4), and the number of channels shown in the table becomes 64. The appearance characteristic information is extracted as shallow stages from the 1 st stage ConvX1 and the 2 nd stage ConvX2, and thus one convolution layer is used in order to pursue efficiency.
The structures of Stage3 layer, stage4 layer and Stage5 layer are as shown in fig. 4, and the structure comprises two STDC modules with 1 stride and 2 stride, wherein each of the STDC modules with 1 stride and 2 stride comprises block1, block2, block3 and block4 which are sequentially connected in series, and the outputs of block1, block2, block3 and block4 are sent to fusion. Block1 is ConvX with a convolution kernel of 1×1, and Block2-4 is ConvX with a convolution kernel of 3×3, with ConvX containing convolution (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combining operations. The number of channels of blocks 1-3 is 1/2 pi, i is the block sequence number, and the channel of block4 is consistent with the channel of block 3. For the STDC module with the stride of 2, the convolution kernel stride of block2 is 2, and in order to achieve the consistent size of the feature map during fusion, the output of block1 is subjected to averaging pooling with the stride of 2 and then transmitted to fusion.
The feature map resolution is reduced by half with each Stage, and the final resolution= (H/32, w/32).
Stage 6 generates a signature with large receptive field by a ConvX6 layer, a global average pooling layer.
In particular, how the decoder effectively fuses the multi-level features besides the road information features can be extracted faster and better by using the STDC network is also a key for realizing high segmentation precision. In this application, feature representation is effectively enhanced in distinction to the unified attention fusion module proposed in the existing PPLiteSeg model. The present application uses a multi-attention fusion module in a decoder, which applies channels and spatial attention to enrich the fusion feature representation, the structure of which is shown in fig. 6.
As described in connection with fig. 3 and 6, for a multi-attention fusion module, the input features are two-part input features denoted as F high And F low 。F high Is the output of the decoder's own module, F low Is the output of the corresponding module of the encoder; corresponding F high And F low In F high Is of the size F low Size (1/2 ). During operation, the multi-attention fusion module first uses bilinear interpolation to interpolate F high Upsampling to AND F low The same size, and the upsampling feature is denoted as F up Then, the multi-attention fusion module uses F up And F low As input, the attention weight is calculated by using a mixed attention mechanism which integrates a channel attention module and a space attention module in sequence, and the efficiency and the performance of the visual task of the computer are improved by applying the mixed attention mechanism. After carrying out maximum pooling and average pooling on the input respectively, a channel attention module of the mixed attention mechanism respectively inputs the maximum pooling and average pooling results into an MLP, and two characteristics after MLP processing are added and activated through a sigmod function; the spatial attention module of the mixed attention mechanism averages and maximizes the input from the channel dimension, combines the averaged and maximized results, and then activates by a convolution and sigmod function. After the attention weights alpha are obtained, fu is respectively carried out p And the attention weights alpha, F low And (1-attention weight alpha) performing element-by-element multiplication operation to realize attention weighting. Finally, carrying out element-by-element addition operation on the weighted attention characteristic to obtain F out 。
The operation of the multi-attention fusion module implementation can be expressed by the following formula:
F up ＝Upsample(F high )；
u＝sigmod(MLP(maxpool(F up F lov ))+MLP(avgpool(F up ，F low )))，
α＝sigmod(conv(cat([mean(u)，avg(u)])))；
F out ＝F up ·α+F ow (1-alpha). Wherein cat represents fusion, mean represents torch.mean function, avg represents torch.avg function.
The training process adopts a gradient descent (SGD) optimization strategy, the batch size batch_size of the data set is set to 8 in each training, and the initial learning rate is 0.01; meanwhile, the first 3 rounds use a linear learning rate warming strategy, the linear learning rate warming is to initially adjust the learning rate, and the learning rate is gradually increased from small to large before the learning rate is normally adjusted. After each super parameter is well adjusted, model training is started.
S3: the road segmentation model of the trained road segmentation transmission channel is deployed into monitoring equipment of the transmission line, the road region is accurately segmented by using the road segmentation model, and the segmentation result is linked with the detection result of the detection model to accurately divide the alarm level. The method comprises the following specific steps:
s31: the trained road segmentation power transmission channel road segmentation model is deployed into monitoring equipment, and the pixel region coordinates of the road are obtained through analysis of the image which is shot by monitoring equipment, and the road region recognition result is shown in fig. 7.
S32: and detecting the hidden danger of interest through the deployed power transmission channel detection model, and feeding back the coordinates of the key hidden danger.
S33: and (3) carrying out alarm grade division on the hidden danger by calculating the relation between the coordinates of the pixel area of the road and the coordinates of the key hidden danger, such as IoU, wherein different alarm grades correspond to different processing methods. Namely, the alarm levels are classified into severe alarm, medium alarm and no alarm according to IoU of the two areas of the segmentation result and the detection result of the detection model.
Example 2
Referring to fig. 8, an embodiment of the present invention provides a road abnormality warning system in a power transmission channel scene, to implement a road abnormality warning method in the power transmission channel scene, including:
the image acquisition module is used for acquiring scene images of the power transmission channel in real time or at fixed time through the monitoring camera;
the power transmission channel road segmentation module is used for identifying and segmenting the roads in the power transmission channel scene image based on the power transmission channel road segmentation model; the transmission channel road segmentation model is an improved PPLiteSeg model. The modified PPLiteSeg model employs a codec architecture, and the network architecture of the STDC network of the encoder is shown in fig. 5, including ConvX1 layer, convX2 layer, stage3 layer, stage4 layer, stage5 layer, convX6 layer, and GlobalPool layer, where ConvX (x=1, 2, 6) contains a convolution (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combining operation. Stage3, stage4, and Stage5 are all STDC modules forming an STDC network. The present application uses a multi-attention fusion module in a decoder that applies channel and spatial attention to enrich the fusion feature representation. During operation, the multi-attention fusion module first uses bilinear interpolation to interpolate F high Upsampling to AND F low The same size, and the upsampling feature is denoted as F up Then, the multi-attention fusion module uses F up And F low As input, the attention weight is calculated by using a mixed attention mechanism which integrates a channel attention module and a space attention module in sequence, and the efficiency and the performance of the visual task of the computer are improved by applying the mixed attention mechanism. After carrying out maximum pooling and average pooling on the input respectively, a channel attention module of the mixed attention mechanism respectively inputs the maximum pooling and average pooling results into an MLP, and two characteristics after MLP processing are added and activated through a sigmod function; the spatial attention module of the mixed attention mechanism averages and maximizes the input from the channel dimension, combines the averaged and maximized results, and then activates by a convolution and sigmod function. After obtaining the attention weights alpha, F is respectively carried out up And the attention weights alpha, F low And (1-attention weight alpha) performing element-by-element multiplication operation to realize attention weighting. Finally, the injection is carried outThe weighted characteristics of the intention are added element by element to obtain F out 。
The detection module is used for detecting the abnormality of the scene image of the power transmission channel;
and the alarm module is used for accurately dividing the alarm level by linking the segmentation result with the detection result of the detection model for detecting the abnormality.
Example 3
The embodiment of the invention provides a computer readable storage medium, which stores a computer program, and when the computer program is executed, the method for warning of road abnormality in a power transmission channel scene is realized, and comprises the following steps:
s1: acquiring a power transmission channel scene image of a road, using an X-AnyLabeling tool, and carrying out pixel-level polygonal labeling on a road region in the power transmission channel scene image by using a SAM model in a semi-automatic manner to construct a road segmentation data set in the power transmission channel scene;
s2: carrying out segmentation training on the improved PPLiteSeg model by using a road segmentation data set to finally obtain a transmission channel road segmentation model; the modified PPLiteSeg employs a codec architecture, and the network architecture of the STDC network of the encoder is shown in fig. 5, including ConvX1 layer, convX2 layer, stage3 layer, stage4 layer, stage5 layer, convX6 layer, and GlobalPool layer, where ConvX (x=1, 2, 6) contains a convolution (Conv) -Batch Normalization (BN) -ReLU activation (ReLU) combining operation. Stage3, stage4, and Stage5 are all STDC modules forming an STDC network. The present application uses a multi-attention fusion module in a decoder that applies channel and spatial attention to enrich the fusion feature representation. During operation, the multi-attention fusion module first uses bilinear interpolation to interpolate F high Upsampling to AND F low The same size, and the upsampling feature is denoted as F up Then, the multi-attention fusion module uses F up And F low As input, attention weights are calculated using a hybrid attention mechanism that integrates a channel attention module and a spatial attention module in succession, improving computer vision by applying the hybrid attention mechanismEfficiency and performance of tasks. After carrying out maximum pooling and average pooling on the input respectively, a channel attention module of the mixed attention mechanism respectively inputs the maximum pooling and average pooling results into an MLP, and two characteristics after MLP processing are added and activated through a sigmod function; the spatial attention module of the mixed attention mechanism averages and maximizes the input from the channel dimension, combines the averaged and maximized results, and then activates by a convolution and sigmod function. After obtaining the attention weights alpha, F is respectively carried out up And the attention weights alpha, F low And (1-attention weight alpha) performing element-by-element multiplication operation to realize attention weighting. Finally, carrying out element-by-element addition operation on the weighted attention characteristic to obtain F out 。
S3: the trained transmission channel road segmentation model is deployed into monitoring equipment of the transmission line, the road area where the monitoring equipment collects images is segmented through the transmission channel road segmentation model, and the segmentation result is linked with the detection result of the detection model to accurately divide the alarm level.
Of course, the computer readable storage medium provided by the embodiment of the present invention stores a computer program not limited to the method operations described above, but also can execute the related operations in the road abnormality warning method in the power transmission channel scene provided by any embodiment of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed structures and methods may be implemented in other manners. For example, the structural embodiments described above are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via interfaces, structures or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. The road abnormality warning method in the scene of the transmission channel is characterized by comprising the following steps:
s1: acquiring a power transmission channel scene image of a road, using an X-AnyLabeling tool, and carrying out pixel-level polygonal labeling on a road region in the power transmission channel scene image by using a SAM model in a semi-automatic manner to construct a road segmentation data set in the power transmission channel scene;
s2: carrying out segmentation training on the improved PPLiteSeg model by using a road segmentation data set to finally obtain a transmission channel road segmentation model;
s3: the trained transmission channel road segmentation model is deployed into monitoring equipment of a transmission line, a road area where the monitoring equipment collects images is segmented through the transmission channel road segmentation model, and the segmentation result is linked with a detection result of a detection model for detecting abnormality to accurately divide the alarm level.
2. The method for warning of abnormal road conditions in a power transmission path scene according to claim 1, wherein the step S1 of semi-automatically labeling the power transmission path road scene image, the construction of the road segmentation data set comprises:
acquiring a power transmission channel scene image containing a road area under a power transmission channel scene through visual monitoring equipment which is already installed on a power transmission line;
performing pixel-level polygonal labeling on the road area by using an X-AnyLabeling tool and a SAM model in a semi-automatic manner, wherein the label is read, and obtaining a json format file with labeled completion;
the marked files are processed according to the following steps of 7:1: and 2, dividing the ratio into a training set, a verification set and a test set to obtain a road segmentation data set.
3. The method of claim 1, wherein the transmission path road segmentation model uses an STDC network as a backup for the whole transmission path road segmentation model to extract road features in the transmission scene, the STDC network of the encoder of the modified PPLiteSeg model comprising a ConvX1 layer, a ConvX2 layer, a Stage3 layer, a Stage4 layer, a Stage5 layer, a ConvX6 layer, and a GlobalPool layer, wherein ConvX, x = 1,2,6 comprise a combination of convolutional-batch normalization-ReLU activation; stage3, stage4 and Stage5 are all STDC modules forming an STDC network, and the GlobalPool layer is a global average pooling layer.
4. The method for warning of abnormal road conditions in a power transmission channel scene according to claim 3, wherein the structure of Stage3 layer, stage4 layer and Stage5 layer comprises two STDC modules with steps of 1 and 2, each of the STDC modules with steps of 1 and 2 comprises block1, block2, block3 and block4 connected in series in sequence, and outputs of block1, block2, block3 and block4 are sent to fusion; block1 is ConvX with convolution kernel of 1×1, block2-4 is ConvX with convolution kernel of 3×3, and ConvX comprises convolution-batch normalization-ReLU activation combination operation; the number of the channels of the blocks 1-3 is 1/2 pi, i is the serial number of the block, and the channel of the block4 is consistent with the channel of the block 3; for the STDC module with the stride of 2, the convolution kernel stride of block2 is 2, and the output of block1 is subjected to averaging pooling with the stride of 2 and then transmitted to fusion.
5. The method of claim 1, wherein a multi-attention fusion module is used in the decoder to apply channel and spatial attention to enrich the fusion feature representation; the multi-attention fusion module first uses bilinear interpolation to input F high Upsampling to F with input low Equal size, and F high The up-sampled feature is denoted as F up The multi-attention fusion module is F up And F low As input, a mixed attention mechanism integrating a channel attention module and a space attention module sequentially is used for calculating attention weights; will F up And attention weight, F low And (1-attention weight) performing element-by-element multiplication operation to realize attention weighting, and performing element-by-element addition operation on the attention weighted features to obtain output.
6. The method for warning of abnormal road conditions in a power transmission channel scene according to claim 5, wherein the channel attention module of the mixed attention mechanism performs maximum pooling and average pooling on the input respectively, the maximum pooling and average pooling result is input into the MLP respectively, and the two characteristics after MLP processing are activated by a sigmod function after being added.
7. The method of claim 5, wherein the spatial attention module of the hybrid attention mechanism averages and maximizes the input from the channel dimension, combines the averaged and maximized results, and then activates via a convolution and sigmod function.
8. The method for warning of abnormal road conditions in a power transmission path scenario according to claim 1, wherein the step S3 comprises: and (5) dividing alarm grades according to the segmentation result and the IoU of the detection result of the detection model, wherein the alarm grades are classified into serious alarms, medium alarms and no alarms.
9. A road abnormality warning system in a power transmission channel scene, implementing the road abnormality warning method in a power transmission channel scene as set forth in any one of claims 1 to 8, characterized by comprising:
the image acquisition module is used for acquiring scene images of the power transmission channel in real time or at fixed time through the monitoring camera;
the power transmission channel road segmentation module is used for identifying and segmenting the roads in the power transmission channel scene image based on the power transmission channel road segmentation model;
the detection module is used for detecting the abnormality of the scene image of the power transmission channel;
and the alarm module is used for accurately dividing the alarm level by linking the segmentation result with the detection result of the detection model for detecting the abnormality.
10. A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the road anomaly warning method in a power transmission channel scene according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311709006.1A CN117809178A (en) | 2023-12-12 | 2023-12-12 | Road abnormity warning method, system and medium in power transmission channel scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311709006.1A CN117809178A (en) | 2023-12-12 | 2023-12-12 | Road abnormity warning method, system and medium in power transmission channel scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117809178A true CN117809178A (en) | 2024-04-02 |
Family
ID=90428971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311709006.1A Withdrawn CN117809178A (en) | 2023-12-12 | 2023-12-12 | Road abnormity warning method, system and medium in power transmission channel scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117809178A (en) |
-
2023
- 2023-12-12 CN CN202311709006.1A patent/CN117809178A/en not_active Withdrawn
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102094341B1 (en) | System for analyzing pot hole data of road pavement using AI and for the same | |
| Akagic et al. | Pothole detection: An efficient vision based method using rgb color space image segmentation | |
| CN109325915B (en) | Super-resolution reconstruction method for low-resolution monitoring video | |
| CN109543691A (en) | Ponding recognition methods, device and storage medium | |
| CN113688709B (en) | Intelligent detection method, system, terminal and medium for wearing safety helmet | |
| JP6700373B2 (en) | Apparatus and method for learning object image packaging for artificial intelligence of video animation | |
| CN113808098A (en) | Road disease identification method and device, electronic equipment and readable storage medium | |
| CN115223130B (en) | Multi-task panoramic driving perception method and system based on improved YOLOv5 | |
| CN115546742A (en) | Rail foreign matter identification method and system based on monocular thermal infrared camera | |
| CN114693963A (en) | Recognition model training and recognition method and device based on electric power data feature extraction | |
| CN114529890A (en) | State detection method and device, electronic equipment and storage medium | |
| CN112597996B (en) | Method for detecting traffic sign significance in natural scene based on task driving | |
| CN116189063B (en) | Key frame optimization method and device for intelligent video monitoring | |
| JP6681965B2 (en) | Apparatus and method for extracting learning target image for autonomous driving | |
| CN116310748A (en) | Automatic driving scene recovery and automatic driving prototype testing method and system | |
| CN112016542A (en) | Urban waterlogging intelligent detection method and system | |
| CN117809178A (en) | Road abnormity warning method, system and medium in power transmission channel scene | |
| CN115578624A (en) | Agricultural disease and pest model construction method, detection method and device | |
| CN112633089B (en) | Video pedestrian re-identification method, intelligent terminal and storage medium | |
| CN112529881B (en) | Power control cabinet cable anomaly identification method and device | |
| CN114360064A (en) | Office place personnel behavior lightweight target detection method based on deep learning | |
| CN114140429A (en) | Real-time parking space detection method and device for vehicle end | |
| CN117197756B (en) | Hidden danger area intrusion detection method, device, equipment and storage medium | |
| CN115273013B (en) | Lane line detection method, system, computer and readable storage medium | |
| CN116189132B (en) | Training method for target detection model of road information, target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20240402 |