CN114972888A - Communication maintenance tool identification method based on YOLO V5 - Google Patents

Abstract

The invention relates to a YOLO V5-based communication overhaul tool identification method, and belongs to the technical field of communication overhaul tool identification. The method adopts an MOT semi-automatic marking tool and a VoTT visual object marking tool to mark the communication line maintenance tool sample to generate a special data set, and has practical value in the communication field; an attention mechanism is adopted to improve a YOLO V5 model, and the improved YOLO V5 model is used to perform data enhancement and model training on the communication line overhaul tool data set, so that the detection efficiency and the identification accuracy are improved; the OpenVINO is adopted to carry out model reasoning optimization and acceleration, so that the detection speed is improved; the communication line maintenance tool is detected in real time and displayed in a result mode by deploying the raspberry serving portable mobile terminal, the problem of identification difficulty of the current communication line maintenance tool is well solved, and convenience is brought to classification, storage and selection of the communication line maintenance tool for communication operation.

Description

Communication maintenance tool identification method based on YOLO V5

Technical Field

The invention belongs to the technical field of communication maintenance tool identification, and particularly relates to a method for identifying a communication maintenance tool based on YOLO V5.

Background

The communication line maintenance tools are various in types, the types relate to program control exchange, network transmission, optical end transmission, image transmission, communication power supply and the like, and the special professional and the complex degree are achieved. The problem that an operator selects a tool improperly in use often occurs in the communication line overhauling process, and the behavior influences the efficiency of 'preemptive generation and communication', and also can cause irreversible damage to a communication line and an interface. In order to reduce the error rate of selecting the communication maintenance tool by the communication operator, the text takes the identification of the communication maintenance tool as a starting point, and provides guidance for the communication operator to correctly select the tool and identify the illegal behavior by accurately identifying the communication maintenance tool. Communication line overhauls the instrument a great deal, and along with information network and optical communication technique and the continuous optimization and the renewal of equipment, communication line overhauls the instrument the kind more and more, brings certain difficulty for discernment and detection. Conventional machine learning related algorithms fail to better identify a significant number of communication line service tools. And the current research aiming at the identification and classification of communication line maintenance tools is less, so that the tools can not be better and conveniently selected by communication operators, and the prompt for finding out the illegal behaviors can not be provided. The current communication technology industry widely refers to deep learning detection technology, wherein the YOLO system algorithm becomes one of important models for engineering application and landing due to good real-time performance and high accuracy.

The method is based on a YOLO V5 algorithm to realize the identification and classification of the electric power tools, performs image enhancement on communication line maintenance tool samples, performs transfer learning on a data set through preprocessing, and further improves the accuracy of electronic maintenance tool identification. The research method is transferred from the field of electric power tools to the field of communication line maintenance tools, a new data set needs to be made, the problem that the recognition rate of the algorithm is reduced in multiple application scenes such as indoor, outdoor, night and strong light is solved, and the method is deployed to a mobile terminal and is convenient to carry and use. Therefore, the characteristics of the application scene, shape, size, color and the like of the communication line detection tool are fully considered, a self-attention mechanism is introduced to improve the YOLO V5 algorithm, and the identification accuracy is improved. In consideration of maneuvering requirements, the improved YoLO V5 algorithm is reasoned and accelerated by OpenVINO and is deployed on a mobile terminal, and a better effect is achieved.

Disclosure of Invention

Technical problem to be solved

The invention aims to solve the technical problem of how to provide a communication overhaul tool identification method based on YOLO V5 so as to solve the problems of reduced identification rate and inconvenience in use in a plurality of application scenes such as indoor, outdoor, night and strong light.

(II) technical scheme

In order to solve the technical problem, the invention provides a communication overhaul tool identification method based on YOLO V5, which comprises the following steps:

s1: carrying out data annotation on video clips and images of the communication overhaul tool by adopting an MOT semi-automatic annotation tool and a VoTT visual object annotation tool to obtain a communication overhaul tool data set;

s2: on the basis of a depth separable residual error network structure, an SE, CA or CBAM attention mechanism is introduced, and a YOLO V5 backbone network is improved;

s3: carrying out image enhancement on the communication overhaul tool data set by using the improved YOLO V5 backbone network, and carrying out model training to obtain trained model parameters and weights;

s4: reasoning and performance optimization are carried out on the trained YOLO V5 backbone network by using an OpenVINO optimizer;

s5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the category of the corresponding communication line maintenance tool.

Further, the step S1 specifically includes: the method comprises the steps that video clips and pictures of a communication line maintenance tool are collected through camera equipment to form samples, the MOT semi-automatic marking tool is used for carrying out semi-automatic marking on the video clips of the communication maintenance tool, the VoTT visual object marking tool is used for carrying out data marking on the pictures of the communication maintenance tool, and two marking results are collected to obtain a communication maintenance tool data set.

Further, the step S2 specifically includes the following steps: one of three network components selected from SENEt (Squeeze-and-Excitation Networks), CA (channel attachment), CBAM (volumetric Block attachment Module), CSP1_1 network component replacing the backhaul part of the YOLO V5 network structure;

for the SENET network component, a channel attention mechanism is introduced, channel dimensions are optimized through two steps of extrusion and excitation, a small number of parameters are added, and features on different channels are acquired by a YOLO V5 backbone network;

for a CA network component, decomposing the channel attention into two 1-dimensional feature coding processes, aggregating features along 2 spatial directions respectively, and acquiring remote dependency relations and retaining accurate position information respectively;

for a CBAM network component, the SENEt and the CA are connected in series as two independent components, and a space attention mechanism and a channel attention mechanism are comprehensively considered.

Further, the S3 specifically includes the following steps:

s31: the improved YOLO V5 backbone network adopts a Mosaic data enhancement method to perform feature enhancement on the communication overhaul tool image after the annotation, so as to obtain enhanced image data;

s32: dividing a communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1;

s33: when calculating the improved YOLO V5 backbone network, there are 3 loss functions: BCE, Focal loss and Qelastic loss, comparing the effects of the 3 loss functions on the aspects of training and detection, and selecting the optimal one;

s34: and training the communication detection tool data set by using the improved YOLO V5 backbone network, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

Further, the BCE loss function is:

wherein

Representing the probability that the ith sample of the backbone network of YOLO V5 is a certain class, wherein N is the total amount of the samples; y (i) represents the true category of the ith sample, and takes the value of 0 or 1.

Further, the Focal loss function is:

wherein p represents the prediction probability, the larger the value of p is, the closer to the category y is, namely, the more accurate the classification is; gamma is an introduced hyper-parameter, the value range is between [0 and 5], the loss values of a positive sample with higher probability and a negative sample with lower probability can be obviously reduced, and the distinguishing capability of the model to difficult samples is improved.

Further, the QFocal loss function is:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein sigma represents a prediction result, the probability value of each picture corresponding to a certain class is from 0 to 1, and beta is a proportionality coefficient.

Further, the feature enhancement includes increasing noise, changing luminance, and changing chrominance.

Further, the step S4 specifically includes the following steps:

s41: the improved pytorech model file of the backbone network of the YOLO V5 is converted into a universal open model format ONNX, the ONNX format file is further converted through a model conversion script of OpenVINO, and IR model files, namely a bin file and an xml file, are generated;

s42: model analysis and calling are completed based on OpenVINO SDK, input and output formats are set, images input by a YOLO V5 are normalized to be between 0 and 1 and are in RGB channel sequence, input and output format data are set to be floating point numbers, input image data are set, inference prediction is achieved, and an optimized YOLO V5 backbone network is obtained.

Further, the step S5 specifically includes the following steps:

s51: the method comprises the steps of improving an operation interface, adding three buttons of a detection picture, a detection video and a detection camera, obtaining an image source through the three ways, detecting, and uniformly converting an image of a communication line maintenance tool to be detected into an image with 640 × 640 resolution;

s52: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

(III) advantageous effects

The invention provides a communication overhaul tool identification method based on YOLO V5,

drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is a diagram of a modified YOLO V5 network architecture;

fig. 3 is an internal structural view of the attention mechanism.

Detailed Description

In order to make the objects, contents and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.

The invention provides a YOLO V5-based communication overhaul tool identification method, which can better solve the identification problem of the current communication line overhaul tool and provide convenience for classified storage and selection of communication line overhaul tool identification for communication operation.

The invention provides a method for identifying a communication overhaul tool based on YOLO V5, which comprises the steps of marking a communication line overhaul tool sample by adopting an MOT semi-automatic marking tool and a VoTT visual object marking tool to generate a special data set, and has practical value in the communication field; an attention mechanism is adopted to improve a YOLO V5 model, and the improved YOLO V5 model is used to perform data enhancement and model training on the communication line overhaul tool data set, so that the detection efficiency and the identification accuracy are improved; OpenVINO is adopted to carry out model reasoning optimization and acceleration, so that the detection speed is improved; the communication line maintenance tool is detected in real time and displayed in a result mode by deploying the raspberry serving portable mobile terminal, the problem of identification difficulty of the current communication line maintenance tool is well solved, and convenience is brought to classification, storage and selection of the communication line maintenance tool for communication operation.

In order to achieve the technical effects, the technical scheme of the invention is as follows:

a communication overhaul tool identification method based on YOLO V5 comprises the following steps:

s1: carrying out data annotation on video clips and images of the communication overhaul tool by adopting an MOT semi-automatic annotation tool and a VoTT visual object annotation tool to obtain a communication overhaul tool data set;

s2: three attention mechanisms of SE, CA and CBAM are introduced based on a depth separable residual error network structure, and a YOLO V5 backbone network is improved;

s3: carrying out image enhancement on the communication overhaul tool data set by using the improved YOLO V5 backbone network, and carrying out model training to obtain trained model parameters and weights;

s4: reasoning and performance optimization are carried out on the trained YOLO V5 backbone network by using an OpenVINO optimizer;

s5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the category of the corresponding communication line maintenance tool.

Further, the step S1 specifically includes the following steps:

the method comprises the steps that video clips and pictures of a communication line maintenance tool are collected through camera equipment to form samples, the MOT semi-automatic marking tool is used for carrying out semi-automatic marking on the video clips of the communication maintenance tool, the VoTT visual object marking tool is used for carrying out data marking on the pictures of the communication maintenance tool, and two marking results are collected to obtain a communication maintenance tool data set.

Further, the step S2 specifically includes the following steps:

CSP1_1 network component (512 × 20 × 20) replacing the backhaul part of the YOLO V5 network structure, which is any one of three network components from SENet (Squeeze-and-requirements networks), ca (channelantentation), cbam (volumetric blockattentionandmodule):

for the SENET network component, a channel attention mechanism is introduced, channel dimensions are optimized through two steps of extrusion and excitation, a small number of parameters are increased, and features on different channels are acquired by a YOLO V5 backbone network, so that the accuracy is improved;

for the CA network component, the channel attention is decomposed into two 1-dimensional feature coding processes, features are aggregated along 2 spatial directions respectively, and remote dependency relationships and accurate position information are acquired and reserved respectively, so that feature representation of target detection is enhanced;

for a CBAM network component, the SENEt and the CA are connected in series as two independent sub-modules, and a space attention mechanism and a channel attention mechanism are comprehensively considered.

Further, the S3 specifically includes the following steps:

s31: the improved YOLO V5 backbone network adopts a Mosaic data enhancement method to perform feature enhancement on the communication overhaul tool image after the annotation, so as to obtain enhanced image data;

s32: dividing a communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1;

s33: the calculation of the improved YOLO V5 backbone network can adopt 3 loss functions: BCE, Focal loss, Qnocal loss. The effectiveness of these 3 loss functions in training and detection is compared and the best is selected.

S34: and training the communication detection tool data set by using the improved YOLO V5 backbone network, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished. The total amount of samples N is 5000, and the categories of target detection are 7 types: program control exchange, network test, time unification, image transmission, optical fiber transmission, voice communication and communication power supply. The target parameters and the target weights are parameters and weights used in a YOLO V5 backbone network, generally tens of millions, and need to be placed in a model file.

Further, the BCE loss function is:

wherein

Representing the probability that the ith sample of the YOLO V5 backbone network is of a certain class, and N is the total number of samples. y (i) represents the true category of the ith sample, and takes the value of 0 or 1.

Further, the Focal loss function is:

wherein p represents the prediction probability, the larger the value of p is, the closer to the category y is, namely, the more accurate the classification is; p also reflects the difficulty of classification, and the larger the value of p is, the higher the confidence of classification is, which means that the sample is easier to distinguish; the smaller its value, the lower the confidence of the classification, meaning the more difficult the sample to resolve. Gamma is an introduced hyper-parameter, the value range is between [0,5], the loss values of the positive sample with higher probability and the negative sample with lower probability can be obviously reduced, and the distinguishing capability of the model to the difficult sample is improved.

Further, the QFocal loss function is:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein sigma represents a prediction result, the probability value of each picture corresponding to a certain class is from 0 to 1, and beta is a proportionality coefficient.

Further, the step S4 specifically includes the following steps:

s41: the improved pytorech model file of the backbone network of the YOLO V5 is converted into a universal open model format ONNX, the ONNX format file is further converted through a model conversion script of OpenVINO, and IR model files, namely a bin file and an xml file, are generated;

s42: model analysis and calling are completed based on OpenVINO SDK, input and output formats are set, images input by a YOLO V5 are normalized to be between 0 and 1 and are in RGB channel sequence, input and output format data are set to be floating point numbers, input image data are set, inference prediction is achieved, and an optimized YOLO V5 backbone network is obtained.

Further, the step S5 specifically includes the following steps:

s51: the method comprises the steps of improving an operation interface, adding three buttons of a detection picture, a detection video and a detection camera, obtaining an image source through the three ways, detecting, and uniformly converting an image of a communication line maintenance tool to be detected into an image with 640 × 640 resolution;

s52: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

Further, the feature enhancement includes increasing noise, changing luminance, and changing chrominance; beta is 2.

The drawings are for illustration purposes only and are not to be construed as limiting the present patent;

for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;

it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.

The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.

Example 1

A communication overhaul tool identification method based on YOLO V5 comprises the following steps:

s1: carrying out data annotation on video clips and images of the communication overhaul tool by adopting an MOT semi-automatic annotation tool and a VoTT visual object annotation tool to obtain a communication overhaul tool data set;

s2: three attention mechanisms of SE, CA and CBAM are introduced based on a depth separable residual error network structure, and a YOLO V5 backbone network is improved;

s3: carrying out image enhancement on the communication overhaul tool data set by using the improved YOLO V5 backbone network, and carrying out model training to obtain trained model parameters and weights;

s4: reasoning and performance optimization are carried out on the trained YOLO V5 backbone network by using an OpenVINO optimizer;

and S5, deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the type of the corresponding communication line maintenance tool.

Step S1 specifically includes the following steps:

the method comprises the steps that video clips and pictures of a communication line maintenance tool are collected through camera equipment to form samples, the MOT semi-automatic labeling tool is used for carrying out semi-automatic labeling on the video clips of the communication maintenance tool, the VoTT visual object labeling tool is used for carrying out data labeling on the pictures of the communication maintenance tool, the two labeling results are collected to obtain 5000 high-quality samples, and a communication maintenance tool data set is formed.

Step S2 specifically includes the following steps:

CSP1_1 network component (512 × 20 × 20) replacing the backhaul part of the YOLO V5 network structure, which is selected from three network components, SENet (Squeeze-and-Excitation Networks), ca (channelantent), cbam (volumetric block attachment module):

for the SENET network component, a channel attention mechanism is introduced, channel dimensions are optimized through two steps of extrusion and excitation, a small number of parameters are increased, and features on different channels are acquired by a YOLO V5 backbone network, so that the accuracy is improved;

for the CA network component, the channel attention is decomposed into two 1-dimensional feature coding processes, features are aggregated along 2 spatial directions respectively, and remote dependency relationships and accurate position information are acquired and reserved respectively, so that feature representation of target detection is enhanced;

for a CBAM network component, the SENEt and the CA are connected in series as two independent sub-modules, and a space attention mechanism and a channel attention mechanism are comprehensively considered.

Step S3 specifically includes the following steps:

s31: the improved YOLO V5 backbone network adopts a Mosaic data enhancement method to perform feature enhancement on the communication overhaul tool image after the annotation, so as to obtain enhanced image data;

s32: dividing a communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1;

s33: the calculation of the improved YOLO V5 backbone network can adopt 3 loss functions: BCE, Focal loss, Qnocal loss. The effectiveness of these 3 loss functions in training and detection is compared and the best is selected.

S34: and training the communication detection tool data set by using the improved YOLO V5 backbone network, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

The BCE loss function is:

wherein

Representing the probability that the ith sample of the YOLO V5 backbone network is of a certain class.

The Focal loss function is:

wherein p is _t Indicates the prediction probability, the larger the value thereof isThe closer to category y the better, i.e. the more accurate the classification; gamma is an introduced hyper-parameter, so that the loss values of the positive sample with higher probability and the negative sample with lower probability are obviously reduced, and the distinguishing capability of the model on the difficult samples is improved.

The Qnocal loss function is:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein the probability value of each picture corresponding to a certain class is from 0 to 1.

Step S4 specifically includes the following steps:

s41: the improved pytorech model file of the backbone network of the YOLO V5 is converted into a universal open model format ONNX, the ONNX format file is further converted through a model conversion script of OpenVINO, and IR model files, namely a bin file and an xml file, are generated;

s42: model analysis and calling are completed based on OpenVINO SDK, input and output formats are set, images input by YOLO V5 are normalized to be between 0 and 1, the images are in RGB channel sequence, input and output format data are set to be floating point numbers, input image data are set, inference prediction is achieved, and an optimized YOLO V5 backbone network is obtained.

Step S5 specifically includes the following steps:

s51: the method comprises the steps of improving an operation interface, adding three buttons of a detection picture, a detection video and a detection camera, obtaining an image source through the three ways, detecting, and uniformly converting an image of a communication line maintenance tool to be detected into an image with 640 × 640 resolution;

s52: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

Example 2

As shown in FIG. 1, a communication service tool identification method based on YOLO V5 comprises the following steps:

s1: collecting sample image data of a communication line maintenance tool and marking the sample image data; s1 specifically includes the following steps: collecting sample image data of a communication line maintenance tool; and carrying out semi-automatic marking on the video clip of the communication overhaul tool by utilizing the MOT semi-automatic marking tool. And carrying out data annotation on the pictures of the communication overhaul tool by using a VoTT visual object annotation tool. And summarizing the two labeling results to obtain about 5000 samples with high quality, and forming a communication overhaul tool data set.

S2: three attention mechanisms of SE, CA and CBAM are introduced, and a YOLO V5 backbone network is improved, and the method comprises the following steps with reference to FIGS. 2 and 3:

s201: as shown in fig. 2, in the backhaul part of the YOLO V5 network structure, the CSP _1 network component can be replaced by three network components, namely SENet, CA and CSMA, and different attention mechanisms are introduced, so that improvement of a YOLO V5 model is realized, and detection accuracy is improved;

s202: as shown in fig. 3, there are 4 network components. A SENet network component introduces a channel attention mechanism, optimizes channel dimensionality and increases a small number of parameters through two steps of extrusion and excitation, so that a model can better acquire characteristics on different channels, and the accuracy is improved. And the CA network component decomposes the channel attention into two 1-dimensional feature coding processes, aggregates features along 2 spatial directions respectively, and acquires remote dependency relationships and retains accurate position information respectively, thereby enhancing feature representation of target detection. The CBAM network component combines the SENEt and the CA as two independent sub-modules together, comprehensively considers a space attention mechanism and a channel attention mechanism, and is a mixed attention mechanism;

s3: carrying out image enhancement on a communication overhaul tool data set by using the improved model, and carrying out model training to obtain trained model parameters and weights, wherein the method specifically comprises the following steps:

s301: the improved YOLO V5 model performs feature enhancement on the annotated communication overhaul tool image by adopting a Mosaic data enhancement method to obtain enhanced image data, wherein the feature enhancement comprises noise increase, brightness change and chroma change;

the method for increasing the noise comprises the following steps: the number of the pixels which increase the noise is randomly selected to be 1/10 total pixels, and Gaussian noise coverage is carried out on the total pixels, so that excessive loss of characteristics can be avoided under the condition of properly increasing the noise; the method for changing the brightness is as follows: carrying out random brightness change on the image, wherein the random proportion of the brightness change is 0.5, 0.75, 1.25 and 1.5, so as to simulate the change of the illumination intensity in the real working environment; the method for changing the chromaticity is as follows: random chromaticity transformation is carried out on the image, RGB three channels of the image are adjusted according to 0.5, 1 and 1.5 respectively at random, and then the channels are changed, so that physical phenomena such as illumination refraction and reflection in a real environment are simulated;

s302: and dividing the communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1. The images folder and the labels folder respectively comprise three folders of train, val and test, which respectively represent a training set, a verification set and a test set. Storing pictures in three folders in images folders, and storing correspondingly labeled data labels in yolo format in three folders in labels folders;

s303: the improved YOLO V5 model uses three loss functions: BCE (Binary Cross Engine Loss, Binary Cross Entropy), Focal loss, Qnocal loss.

The BCE loss function is:

wherein

Representing the probability that the ith sample of the YOLO V5 backbone network is of a certain class.

The Focal loss function is:

wherein p is _t The prediction probability is represented, and the larger the value of the prediction probability is, the closer the prediction probability is to the category y is, namely, the more accurate the classification is; gamma is an introduced hyper-parameter, so that the loss values of a positive sample with higher probability and a negative sample with lower probability are obviously reduced, and the distinguishing capability of the model on difficult samples is improved。

The Qelastic loss function is as follows:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein, the probability value of each picture corresponding to a certain class is from 0 to 1, and when beta is 2, the effect is best.

S304: and training the communication detection tool data set by using the improved YOLO V5 model, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

S4: the method comprises the following steps of carrying out reasoning and performance optimization on a trained model by using an OpenVINO optimizer:

s401: the modified pytorech model file of the YOLO V5 model is converted into the universal open model format ONNX. Further converting the ONNX format file through the model conversion script of OpenVINO to generate an IR model file, (. bin file and. xml file);

s402: and completing model analysis and calling based on the OpenVINO SDK. The input and output format is set, the image input by YOLO V5 is normalized between 0 and 1, and is in RGB channel order, and the input and output format data is set to floating point numbers. And setting input image data and realizing inference prediction to obtain an optimized YOLO V5 model.

S5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the category of the corresponding communication line maintenance tool. The method specifically comprises the following steps:

s501: the operation interface is improved, three buttons of a detection picture, a detection video and a detection camera are added, and the image source can be obtained and detected through the three ways. Uniformly converting the image of the communication line maintenance tool to be tested into an image with 640 x 640 resolution;

s502: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

Example 3

As shown in FIG. 1, a communication service tool identification method based on YOLO V5 comprises the following steps:

s1: collecting sample image data of a communication line maintenance tool and marking the sample image data; s1 specifically includes the following steps: collecting sample image data of a communication line maintenance tool; and carrying out semi-automatic marking on the video clip of the communication overhaul tool by utilizing the MOT semi-automatic marking tool. And carrying out data annotation on the picture of the communication overhaul tool by using a VoTT visual object annotation tool. Summarizing the two labeling results to obtain a communication overhaul tool data set; let the labeled tags total 7 categories: (1) program control exchange, (2) network test, (3) time unification, (4) image transmission, (5) optical fiber transmission, (6) voice communication, and (7) communication power supply;

s2: three attention mechanisms of SE, CA and CBAM are introduced, and a YOLO V5 backbone network is improved, and the method comprises the following steps with reference to FIGS. 2 and 3:

s201: as shown in fig. 2, in the backhaul part of the YOLO V5 network structure, the CSP _1 network component can be replaced by three network components, namely SENet, CA and CSMA, and different attention mechanisms are introduced, so that improvement of a YOLO V5 model is realized, and detection accuracy is improved;

s202: as shown in fig. 3, there are 4 network components. A SENet network component introduces a channel attention mechanism, optimizes channel dimensionality and increases a small number of parameters through two steps of extrusion and excitation, so that a model can better acquire characteristics on different channels, and the accuracy is improved. And the CA network component decomposes the channel attention into two 1-dimensional feature coding processes, aggregates features along 2 spatial directions respectively, and acquires remote dependency relationships and retains accurate position information respectively, thereby enhancing feature representation of target detection. The CBAM network component combines the SENEt and the CA as two independent sub-modules together, comprehensively considers a space attention mechanism and a channel attention mechanism, and is a mixed attention mechanism;

s3: carrying out image enhancement on a communication overhaul tool data set by using the improved model, and carrying out model training to obtain trained model parameters and weights, wherein the method specifically comprises the following steps:

s301: the improved YOLO V5 model performs feature enhancement on the annotated communication overhaul tool image by adopting a Mosaic data enhancement method to obtain enhanced image data, wherein the feature enhancement comprises noise increase, brightness change and chroma change. The method for increasing the noise is as follows: the number of the pixels which increase the noise is randomly selected to be 1/10 total pixels, and Gaussian noise coverage is carried out on the total pixels, so that the characteristic excessive loss can be avoided under the condition of properly increasing the noise. The method for changing the brightness is as follows: and carrying out random brightness change on the image, wherein the random proportion of the brightness change is 0.5, 0.75, 1.25 and 1.5, so that the change of the illumination intensity in the real working environment is simulated. The method for changing the chromaticity is as follows: random chromaticity transformation is carried out on the image, RGB three channels of the image are adjusted according to 0.5, 1 and 1.5 respectively at random, and then the channels are changed, so that physical phenomena such as illumination refraction and reflection in a real environment are simulated;

s302: and dividing the communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1. The images folder and the labels folder respectively comprise three folders of train, val and test, which respectively represent a training set, a verification set and a test set. Storing pictures in three folders in images folders, and storing correspondingly labeled data labels in yolo format in three folders in labels folders;

s303: the improved YOLO V5 model uses three loss functions: BCE (Binary Cross Engine Loss, Binary Cross Entropy), Focal loss, Qnocal loss.

The BCE loss function is:

wherein

Representing the probability that the ith sample of the YOLO V5 backbone network is of a certain class.

The Focal loss function is:

wherein p is _t The prediction probability is represented, and the larger the value of the prediction probability is, the closer the prediction probability is to the category y is, namely, the more accurate the classification is; gamma is an introduced hyper-parameter, so that the loss values of the positive sample with higher probability and the negative sample with lower probability are obviously reduced, and the distinguishing capability of the model on the difficult samples is improved.

The Qelastic loss function is as follows:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein, the probability value of each picture corresponding to a certain class is from 0 to 1, and when beta is 2, the effect is best.

S304: and training the communication detection tool data set by using the improved YOLO V5 model, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

S4: the method comprises the following steps of carrying out reasoning and performance optimization on a trained model by using an OpenVINO optimizer:

s401: the modified pytorech model file of the YOLO V5 model is converted into the universal open model format ONNX. Further converting the ONNX format file through the model conversion script of OpenVINO to generate an IR model file, (. bin file and. xml file);

s402: and completing model analysis and calling based on the OpenVINO SDK. The input and output format is set, the image input by YOLO V5 is normalized between 0 and 1, and is in RGB channel order, and the input and output format data is set to floating point numbers. And setting input image data and realizing inference prediction to obtain an optimized YOLO V5 model.

S5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the category of the corresponding communication line maintenance tool. The method specifically comprises the following steps:

s501: the operation interface is improved, three buttons of a detection picture, a detection video and a detection camera are added, and the image source can be obtained and detected through the three ways. Uniformly converting the image of the communication line maintenance tool to be tested into an image with 640 x 640 resolution;

s502: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

The optimized model obtained by the invention is used for detecting and comparing the same communication overhaul tool test set with the original model under the conditions of the same training times and the same super parameters. The detection accuracy of the optimized model is improved by 0.9%, and the detection speed is improved by 5.1 times. According to the method, the samples are formed by collecting pictures and videos of the communication line maintenance tool, the training data set is manufactured to train the improved YOLO V5 model, the optimizer is used for reasoning and accelerating, the identification of the communication line maintenance tool is realized, the selection error rate of the communication line maintenance tool is reduced, and the convenience is provided for the classified storage of the communication tool, the selection of the communication operation tool and the identification of illegal behaviors.

The same or similar reference numerals correspond to the same or similar parts;

the positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent;

example 4:

a communication overhaul tool identification method based on YOLO V5 comprises the following steps:

s1: carrying out data annotation on video clips and images of the communication overhaul tool by adopting an MOT semi-automatic annotation tool and a VoTT visual object annotation tool to obtain a communication overhaul tool data set;

s2: three attention mechanisms of SE, CA and CBAM are introduced based on a depth separable residual error network structure, and a YOLO V5 backbone network is improved;

s3: carrying out image enhancement on the communication overhaul tool data set by using the improved YOLO V5 backbone network, and carrying out model training to obtain trained model parameters and weights;

s4: reasoning and performance optimization are carried out on the trained YOLO V5 backbone network by using an OpenVINO optimizer;

s5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the category of the corresponding communication line maintenance tool.

Further, the step S1 specifically includes the following steps:

the method comprises the steps that a video clip and a picture of a communication line maintenance tool are collected through a camera device to form a sample, the MOT semi-automatic marking tool is used for carrying out semi-automatic marking on the video clip of the communication maintenance tool, the VoTT visual object marking tool is used for carrying out data marking on the picture of the communication maintenance tool, and two marking results are summarized to obtain a communication maintenance tool data set.

Further, the step S2 specifically includes the following steps:

the CSP _1 network component of the Backbone part of the YOLO V5 network structure is replaced by three network components of SENET, CA and CSMA respectively:

for the SENET network component, a channel attention mechanism is introduced, channel dimensions are optimized through two steps of extrusion and excitation, a small number of parameters are increased, and features on different channels are acquired by a YOLO V5 backbone network, so that the accuracy is improved;

for the CA network component, the channel attention is decomposed into two 1-dimensional feature coding processes, features are aggregated along 2 spatial directions respectively, and remote dependency relationships and accurate position information are acquired and reserved respectively, so that feature representation of target detection is enhanced;

for a CBAM network component, the SENEt and CA are combined together as two independent sub-modules, and a spatial attention mechanism and a channel attention mechanism are comprehensively considered.

Further, the S3 specifically includes the following steps:

s31: the improved YOLO V5 backbone network adopts a Mosaic data enhancement method to perform feature enhancement on the communication overhaul tool image after the annotation, so as to obtain enhanced image data;

s32: dividing a communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1;

s33: calculating three loss functions BCE, Focal loss and Qnocal loss of the improved YOLO V5 backbone network;

s34: and training the communication detection tool data set by using the improved YOLO V5 backbone network, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

Further, the BCE loss function is:

wherein

Representing the probability that the ith sample of the YOLO V5 backbone network is of a certain class.

Further, the Focal loss function is:

wherein p is _t The prediction probability is represented, and the larger the value of the prediction probability is, the closer the prediction probability is to the category y is, namely, the more accurate the classification is; gamma is an introduced hyper-parameter, so that the loss values of the positive sample with higher probability and the negative sample with lower probability are obviously reduced, and the distinguishing capability of the model on the difficult samples is improved.

Further, the QFocal loss function is:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein the probability value of each picture corresponding to a certain class is from 0 to 1.

Further, the step S4 specifically includes the following steps:

s41: the improved pytorech model file of the backbone network of the YOLO V5 is converted into a universal open model format ONNX, the ONNX format file is further converted through a model conversion script of OpenVINO, and IR model files, namely a bin file and an xml file, are generated;

s42: model analysis and calling are completed based on OpenVINO SDK, input and output formats are set, images input by a YOLO V5 are normalized to be between 0 and 1 and are in RGB channel sequence, input and output format data are set to be floating point numbers, input image data are set, inference prediction is achieved, and an optimized YOLO V5 backbone network is obtained.

Further, the step S5 specifically includes the following steps:

s51: the method comprises the steps of improving an operation interface, adding three buttons of a detection picture, a detection video and a detection camera, obtaining an image source through the three ways, detecting, and uniformly converting an image of a communication line maintenance tool to be detected into an image with 640 × 640 resolution;

s52: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

Further, the feature enhancement includes increasing noise, changing luminance, and changing chrominance; beta is 2.

Compared with the prior art, the technical scheme of the invention has the beneficial effects that:

the invention adopts MOT semi-automatic marking tool and VoTT visual object marking tool to mark the communication line maintenance tool sample, generates special data set, and has practical value in communication field; an attention mechanism is adopted to improve a YOLO V5 model, and the improved YOLO V5 model is used for carrying out data enhancement and model training on the communication line maintenance tool data set, so that the detection efficiency and the identification accuracy are improved; OpenVINO is adopted to carry out model reasoning optimization and acceleration, so that the detection speed is improved; the communication line maintenance tool is detected in real time and displayed in a result mode by deploying the raspberry serving portable mobile terminal, the problem of identification difficulty of the current communication line maintenance tool is well solved, and convenience is brought to classification, storage and selection of the communication line maintenance tool for communication operation.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, it is possible to make various improvements and modifications without departing from the technical principle of the present invention, and those improvements and modifications should be considered as the protection scope of the present invention.

Claims (10)

1. A communication maintenance tool identification method based on YOLO V5 is characterized by comprising the following steps:

s1: carrying out data annotation on video clips and images of the communication overhaul tool by adopting an MOT semi-automatic annotation tool and a VoTT visual object annotation tool to obtain a communication overhaul tool data set;

s2: on the basis of a depth separable residual error network structure, an SE, CA or CBAM attention mechanism is introduced, and a YOLO V5 backbone network is improved;

s3: carrying out image enhancement on the communication overhaul tool data set by using the improved YOLO V5 backbone network, and carrying out model training to obtain trained model parameters and weights;

s4: reasoning and performance optimization are carried out on the trained YOLO V5 backbone network by using an OpenVINO optimizer;

s5: and deploying the optimized model to a mobile terminal of a raspberry group, detecting the image of the communication line maintenance tool subjected to standardized processing, and outputting the type of the corresponding communication line maintenance tool.

2. The YOLO V5-based communication service tool identification method according to claim 1, wherein the step S1 specifically includes: the method comprises the steps that video clips and pictures of a communication line maintenance tool are collected through camera equipment to form samples, the MOT semi-automatic marking tool is used for carrying out semi-automatic marking on the video clips of the communication maintenance tool, the VoTT visual object marking tool is used for carrying out data marking on the pictures of the communication maintenance tool, and two marking results are collected to obtain a communication maintenance tool data set.

3. The YOLO V5-based communication service tool identification method according to claim 1, wherein the step S2 specifically includes the steps of: one of three network components selected from SENEt (Squeeze-and-Excitation Networks), CA (channel attachment), CBAM (volumetric Block attachment Module), CSP1_1 network component replacing the backhaul part of the YOLO V5 network structure;

for the SENET network component, a channel attention mechanism is introduced, channel dimensions are optimized through two steps of extrusion and excitation, a small number of parameters are added, and features on different channels are acquired by a YOLO V5 backbone network;

for a CA network component, decomposing the channel attention into two 1-dimensional feature coding processes, aggregating features along 2 spatial directions respectively, and acquiring remote dependency relations and retaining accurate position information respectively;

for a CBAM network component, the SENEt and the CA are connected in series as two independent components, and a space attention mechanism and a channel attention mechanism are comprehensively considered.

4. The YOLO V5-based communication service tool identification method according to claim 3, wherein the S3 specifically includes the steps of:

s31: the improved YOLO V5 backbone network adopts a Mosaic data enhancement method to perform feature enhancement on the communication overhaul tool image after the annotation, so as to obtain enhanced image data;

s32: dividing a communication overhaul tool image data set into a training set, a verification set and a test set according to the ratio of 3:1: 1;

s33: when calculating the improved YOLO V5 backbone network, there are 3 loss functions: BCE, Focal loss and Qelastic loss, comparing the effects of the 3 loss functions on the aspects of training and detection, and selecting the optimal one;

s34: and training the communication detection tool data set by using the improved YOLO V5 backbone network, and obtaining the target parameters and the target weights identified by the communication line overhaul tool after the training is finished.

5. The YOLO V5-based communication service tool identification method of claim 4, wherein the BCE loss function is:

wherein

Representing the probability that the ith sample of the backbone network of YOLO V5 is a certain class, wherein N is the total amount of the samples; y (i) represents the true category of the ith sample, and takes the value of 0 or 1.

6. The YOLO V5-based communication service tool identification method of claim 4, wherein the Focal loss function is:

FL(p)＝-(1-p _t ) ^γ log(p _t )，

wherein p represents the prediction probability, the larger the value of p is, the closer to the category y is, namely, the more accurate the classification is; gamma is an introduced hyper-parameter, the value range is between [0 and 5], the loss values of a positive sample with higher probability and a negative sample with lower probability can be obviously reduced, and the distinguishing capability of the model to difficult samples is improved.

7. The YOLO V5-based communication service tool identification method of claim 4, wherein the QFocal loss function is:

QFL(σ)＝-|y-σ| ^β ((1-y)log(1-σ)+ylog(σ))

wherein sigma represents a prediction result, the probability value of each picture corresponding to a certain class is from 0 to 1, and beta is a proportionality coefficient.

8. The YOLO V5-based communication service tool identification method of claim 4, wherein the feature enhancements include adding noise, changing brightness, and changing chroma.

9. The YOLO V5-based communication service tool identification method according to any one of claims 4-8, wherein the step S4 specifically includes the steps of:

s41: the improved pytorech model file of the backbone network of the YOLO V5 is converted into a universal open model format ONNX, the ONNX format file is further converted through a model conversion script of OpenVINO, and IR model files, namely a bin file and an xml file, are generated;

s42: model analysis and calling are completed based on OpenVINO SDK, input and output formats are set, images input by a YOLO V5 are normalized to be between 0 and 1 and are in RGB channel sequence, input and output format data are set to be floating point numbers, input image data are set, inference prediction is achieved, and an optimized YOLO V5 backbone network is obtained.

10. The YOLO V5-based communication service tool identification method according to claim 9, wherein the step S5 specifically includes the steps of:

s51: the method comprises the steps of improving an operation interface, adding three buttons of a detection picture, a detection video and a detection camera, obtaining an image source through the three ways, detecting, and uniformly converting an image of a communication line maintenance tool to be detected into an image with 640 × 640 resolution;

s52: and deploying the optimized YOLO V5 model and the converted IR model file to a raspberry party, and displaying, recording and storing the detection result of the communication line maintenance tool in real time.

Images