CN112562698A - Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics - Google Patents
Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics Download PDFInfo
- Publication number
- CN112562698A CN112562698A CN202011404030.0A CN202011404030A CN112562698A CN 112562698 A CN112562698 A CN 112562698A CN 202011404030 A CN202011404030 A CN 202011404030A CN 112562698 A CN112562698 A CN 112562698A
- Authority
- CN
- China
- Prior art keywords
- sound
- information
- generate
- power equipment
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 94
- 238000003745 diagnosis Methods 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004927 fusion Effects 0.000 title claims abstract description 33
- 238000001931 thermography Methods 0.000 title claims abstract description 31
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000009432 framing Methods 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000001228 spectrum Methods 0.000 claims description 28
- 238000004422 calculation algorithm Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 22
- 230000007246 mechanism Effects 0.000 claims description 19
- 239000013598 vector Substances 0.000 claims description 18
- 238000000354 decomposition reaction Methods 0.000 claims description 15
- 238000011176 pooling Methods 0.000 claims description 13
- 230000005236 sound signal Effects 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 10
- 108010076504 Protein Sorting Signals Proteins 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 9
- 230000002068 genetic effect Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 238000002922 simulated annealing Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 230000008901 benefit Effects 0.000 abstract description 8
- 238000005070 sampling Methods 0.000 abstract description 6
- 238000012423 maintenance Methods 0.000 abstract description 5
- 230000001360 synchronised effect Effects 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 description 13
- 230000035772 mutation Effects 0.000 description 13
- 230000006870 function Effects 0.000 description 10
- 230000002159 abnormal effect Effects 0.000 description 8
- 238000010438 heat treatment Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000007614 genetic variation Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000016273 neuron death Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
- G10L19/0216—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation using wavelet decomposition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms
- G06F17/148—Wavelet transforms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/26—Pre-filtering or post-filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Physiology (AREA)
- Genetics & Genomics (AREA)
- Investigating Or Analyzing Materials Using Thermal Means (AREA)
- Radiation Pyrometers (AREA)
Abstract
The application discloses a method for diagnosing defects of electric equipment based on fusion of sound source information and thermal imaging characteristics, which comprises the following steps: acquiring a sample data set of the power equipment, wherein the sample data set at least comprises sound information and a thermal infrared video stream which are subjected to synchronous sampling and framing processing; extracting the characteristics of the sound information, and combining the thermal infrared video stream, and performing characteristic fusion by adopting a convolutional neural network to generate a defect diagnosis model; and training the defect diagnosis model according to the sample data set to determine the operation parameters of the defect diagnosis model, wherein the trained defect diagnosis model is used for defect diagnosis of the power equipment. Through the technical scheme in this application, the diagnostic advantage of full play sound and thermal imaging effectively discerns various defects of power equipment, has improved defect identification efficiency and precision, can in time discover and inform relevant maintenance personal to overhaul, ensures that power equipment is in normal operating condition to the emergence major accident.
Description
Technical Field
The application relates to the technical field of electric power equipment detection, in particular to an electric power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics.
Background
Various power equipment exists in a power grid, different power equipment undertakes respective important tasks, the power equipment has a complex structure and works outdoors for a long time, and various physical and chemical reactions occur inside and outside the power equipment, so that equipment defects are difficult to avoid. With the continuous increase of the degradation degree, the operation of the power grid is unstable, and great economic loss is brought. Therefore, the stability of the power equipment is improved, the normal work of the power equipment is ensured, the power equipment is a very concerned problem, and the research on the power equipment diagnosis technology has great significance.
There are many kinds of defects of the electric power equipment, including thermal defects, electrical defects, etc., and when the operation state of the electric power equipment is defective, different abnormal sounds are generated, which are greatly different from the sound frequency of the normal state, and occasionally heat is generated. The defect detection of the electrical equipment by means of artificial sound listening or thermal imaging observation is greatly influenced by human factors and is only suitable for workers with abundant experience and the condition that the defects are particularly obvious.
With the wide application of the deep learning technology, the defect identification can be realized by converting the sound into the two-dimensional image and putting the two-dimensional image into the convolution network, however, the abnormal sound defect is diagnosed and positioned by adopting the method, the calculation complexity is very high, and even the integrity of the sound source information is influenced. Similarly, when the defect identification is performed through thermal imaging, the method is generally used for diagnosing the heating defect, and only plane information of the defect can be identified, and if the defect without obvious heating phenomenon exists, the identification accuracy rate can be greatly influenced, and the defect judgment is influenced.
However, there is a correlation between abnormal sound and heating defects of the power equipment, and generally abnormal sound and heating characteristics are associated, but at least one phenomenon that characteristics of early defects are not obvious may exist, and if a single signal source is used for defect diagnosis, early defects are easily missed, so that more types of defects of the power equipment can be diagnosed by using a sound and thermal imaging fusion technology.
The existing fusion technology of sound information and thermal imaging is shallow, the existing relatively better method is to adopt a general convolutional neural network CNN to fuse the characteristics of the sound information and the thermal imaging, however, the problems of low integrity of the sound information, high calculation complexity, low identification precision and the like exist, the fusion technology is not found to be used for carrying out defect diagnosis on power equipment, and the defect diagnosis still adopts single-source information to carry out fault diagnosis on the equipment.
Disclosure of Invention
The purpose of this application lies in: the diagnosis advantages of sound and thermal imaging are fully exerted, various defects of the power equipment are effectively identified, defect identification types are increased, the defect identification efficiency and precision are improved, related maintenance personnel can be timely found and notified to overhaul, and the power equipment is ensured to be in a normal operation state so as to avoid major accidents.
The technical scheme of the application is as follows: the method for diagnosing the defect of the electric power equipment based on the fusion of sound source information and thermal imaging characteristics comprises the following steps:
step 1, acquiring a sample data set of the power equipment, wherein the sample data set at least comprises sound information and thermal infrared video streams which are synchronously sampled and subjected to framing processing;
and 3, training the defect diagnosis model according to the sample data set to determine the operation parameters of the defect diagnosis model, wherein the trained defect diagnosis model is used for defect diagnosis of the power equipment.
In any one of the above technical solutions, further, in step 2, the feature extraction of the sound information specifically includes:
step 201, performing frequency domain transformation on sound information by adopting a wavelet packet transformation mode to generate sound frequency spectrum information;
step 202, filtering the sound spectrum information by using a Mel filter bank, and calculating a Mel frequency cepstrum coefficient MFCC corresponding to the filtered sound spectrum information;
step 203, adopting the first-order difference and the second-order difference of the Mel frequency cepstrum coefficient MFCC as the dynamic characteristics of the sound frequency spectrum information, and splicing the dynamic characteristics and the Mel frequency cepstrum coefficient MFCC to generate the sound characteristic parameters of the sound frequency spectrum information;
and step 204, extracting the characteristics of the sound information according to the sound characteristic parameters to generate a sound characteristic vector.
In any of the above technical solutions, further, performing windowing on the framed sound information, step 201, specifically includes:
respectively carrying out decimal coding on wavelet bases and decomposition layer numbers in a wavelet packet conversion mode, determining a value range, and forming double-parameter cascade coding by the two decimal codes to form an initial set;
dividing sound information into a plurality of data points, expressing the data points by using a sound signal sequence, calculating a sequence entropy value E (u) of the sound signal sequence according to double-parameter cascade coding, and converting the sequence entropy value E (u) into fitness, wherein the calculation formula of the fitness Fit (p) is as follows:
Fit(u)＝-E(u)
in the formula I(n,j)Is the nth point data point y(n,j)Signal y decomposed at j-th layerjN represents all data points of the sound signal sequence, and u is the two-parameter cascade coding in the initial set;
and according to the fitness, respectively calculating the optimal wavelet packet basis and the optimal decomposition layer number by adopting a genetic algorithm and a simulated annealing algorithm so as to carry out wavelet packet decomposition on the sound information subjected to frequency division and windowing processing and generate sound frequency spectrum information.
In any one of the above technical solutions, further, in step 203, generating a sound characteristic parameter of the sound spectrum information specifically includes:
taking the first 13 Weimeier frequency cepstrum coefficient MFCC as a characteristic parameter CtA characteristic parameter CtAnd a first order difference dtSecond order difference btThe three are respectively standardized;
respectively calculating characteristic parameters CtFirst order difference dtSecond order difference btThe information entropy of the weight factors of the three;
and calculating the weight factor through the information entropy to generate the sound characteristic parameters of the sound spectrum information.
In any one of the above technical solutions, further, in step 2, in combination with the thermal infrared video stream, performing feature fusion by using a convolutional neural network to generate a defect diagnosis model, specifically including:
step 211, clipping the thermal infrared picture after framing the thermal infrared video stream, and performing channel number expansion to extract a first characteristic diagram of the thermal infrared picture;
step 212, utilizing an attention module mechanism to perform feature extraction on the first feature map to generate a second feature map;
step 213, according to the clipped thermal infrared video stream, establishing a shortcut connection with the second feature map by using an inverse residual error module mechanism, and performing feature extraction by using an attention module mechanism to generate a third feature map;
step 214, according to the third feature map, performing feature extraction by using an attention module mechanism, and establishing a shortcut connection with the third feature map by using an inverted residual module mechanism to generate a fourth feature map;
step 215, performing average pooling operation on the fourth feature map, inputting the fourth feature map to a full connection layer of the convolutional neural network, and generating a thermal image feature vector;
and step 216, performing feature fusion on the thermal image feature vector and the sound feature vector by using a convolutional neural network to generate a defect diagnosis model.
In any one of the above technical solutions, further, in step 212, generating a second feature map specifically includes:
convolving the first characteristic diagram by using different preset convolution kernels to obtain a first branch result Y1And the second branch result Y2；
First branch result Y for different branches1Second branch result Y2Fusing, and generating two weight matrixes a and b through a weighting algorithm and a full connection layer in a convolution network;
performing Softmax activation operation on the two weight matrixes a and b of the last two branches to obtain the weight of each channel in the last two branches, and then obtaining the weight and the result Y of the first branch1Second branch result Y2And multiplying to generate a second feature map.
The beneficial effect of this application is:
according to the technical scheme, a sound feature extraction method for retaining sound source information quantity and reducing calculation complexity is adopted, a few-parameter high-precision convolution neural network is designed to perform feature fusion on extracted sound features and thermal imaging, and finally intelligent fault diagnosis of power equipment adopting multi-source information fusion is achieved.
Compared with the conventional artificial acoustic diagnosis of abnormal sound or heating condition of the power equipment, the method has the advantages that the convolution neural network model is used for fusing the sound characteristic parameters and the thermal imaging characteristics, the collected data is subjected to the automatic defect detection and analysis method by collecting the sound and the thermal imaging video stream of the running state of the power equipment in real time and combining the related algorithms of computer vision and pattern recognition, sound recognition, image processing and the like, and the recognition of various heating and abnormal sound defect types of the power equipment is realized. The advantages of the invention are mainly embodied in the following aspects:
(1) the sound feature extraction effect is good. Compared with the traditional Mel-scale frequency cepstral Coefficients (MFCC), the method avoids manual setting of most parameters and solves the problem that the identification precision of the whole model is influenced by human factors. The optimal wavelet basis and the optimal decomposition layer number of the wavelet packet transformation are obtained by adopting an optimization algorithm, so that the loss of sound source signal information is avoided; and (3) solving the weight factors of the characteristic parameters, the first-order difference and the second-order difference by adopting an information entropy method to obtain the parameters which can reflect the sound characteristics most.
(2) And real-time online diagnosis is realized. When the original method monitors the power equipment, the collected signals are subjected to acoustic diagnosis or thermal diagnosis in an artificial judgment mode, the designed convolutional neural network is adopted to perform feature fusion on the sound features and the thermal features, the attention module and the inverted residual error module are combined, the accuracy of the network is improved, the calculated amount is reduced, real-time monitoring and identification of equipment defects can be performed through the sound sensor and the thermal imaging collection equipment, a warning is timely sent out, maintenance personnel are informed to maintain aiming at the defect reasons, and hidden dangers are eliminated.
(3) The early defect diagnosis sensitivity is high, and the applicability is strong. Compared with the traditional acoustic diagnosis and thermal imaging diagnosis, the method and the system have the advantages that the fault type is judged through abnormal sound and heating representation, effective criterion information is added, the diagnosability of early defects with unobvious characteristics is improved, and early discovery and early operation and maintenance are achieved. The defect diagnosis method and device based on the multi-source judgment acquisition channels can be used for diagnosing defects according to single defect characterization information and/or multi-defect characterization information, and are applicable to defect diagnosis of various power equipment and high in applicability.
(4) The identification precision is high. Compared with other convolutional networks, the designed convolutional neural network is adopted, the sound characteristics and the heat characteristics are subjected to characteristic fusion, the abnormal sound characteristics and the heating characteristics provide rich information, and the multi-scale defect criterion is enhanced, so that defect types with unobvious characteristics can be accurately diagnosed.
Drawings
The advantages of the above and/or additional aspects of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic flow diagram of a method for electrical equipment fault diagnosis based on fusion of acoustic source information with thermal imaging features according to an embodiment of the present application;
FIG. 2 is a schematic flow diagram of thermal imaging and acoustic signal feature fusion according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an attention module mechanism according to one embodiment of the present application;
FIG. 4 is a schematic flow diagram of a multi-objective optimization algorithm according to one embodiment of the present application.
Detailed Description
In order that the above objects, features and advantages of the present application can be more clearly understood, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited by the specific embodiments disclosed below.
As shown in fig. 1, the present embodiment provides a method for diagnosing defects of an electrical device based on fusion of sound source information and thermal imaging characteristics, the method including:
step 1, acquiring a sample data set of the power equipment, wherein the sample data set at least comprises sound information and a thermal infrared video stream which are subjected to synchronous sampling and framing processing, and the sound signal is subjected to windowing processing;
in this embodiment, a microphone is used as the sound information collection device, and the sound information is set to be a single-channel audio with a sampling rate of 16KHz, and includes sample information of the power device in various operating states, that is, a normal operating state and states when various defects occur. Each state takes 5s as training data and 2s as test data, and synchronous sampling is carried out for thermal infrared video streams, and the frame length is 25ms, the frame length is 10ms, and 65 frames per second are carried out on the sound signals.
The thermal infrared imager is adopted as a thermal infrared video stream collecting device, the sampling speed of the thermal infrared imager is set to be 65fps, and the thermal infrared imager and a microphone are synchronously sampled at the same time, so that the sampling data are ensured to be in the same time and the same state. And performing frame processing on the collected thermal infrared video stream, taking 5s as training data and 2s as test data for each state, and manually labeling each frame of thermal imaging.
Further, in order to make the sound change at the beginning and end of the sound information smoother, windowing is performed on the sound information after the framing processing, and the sound information after the framing and windowing processing is used as the sound information in the sample data set, wherein the formula for performing the windowing processing is as follows:
where N is the acoustic signal per frame, i.e. each point in the sequence, N is the length of the sequence, I0The modified bessel function of the first kind is of zero order, and α is any non-negative real number and is used for adjusting the shape of the window, and in this embodiment, α is set to 3/N.
specifically, when the voice information feature extraction is performed, Mel-scale frequency cepstral Coefficients (MFCCs) and the convolutional neural network CNN are generally adopted, however, when the voice feature extraction is performed by using the convolutional neural network CNN, the calculation complexity involved in the extraction process is high, and the amount of information is reduced when the voice original information (voice information) is converted, compared with the case that the conversion manually defined by using the Mel-scale frequency cepstral Coefficients MFCC is close to the optimum in the aspect of voice feature extraction. Moreover, the sound information of the operating state of the power equipment is extremely important, and in order to ensure the accuracy of the sound source information characteristics as much as possible when the sound information is fused with the thermal imaging characteristics, the extraction of the sound characteristics of the power equipment is more suitable for adopting a Mel frequency cepstrum coefficient MFCC.
As shown in fig. 2, the method for extracting features of sound information in this embodiment specifically includes:
step 201, performing frequency domain transformation on sound information by adopting a wavelet packet transformation mode to generate sound frequency spectrum information;
in particular, the sound information varies rapidly and unstably in the time domain, and therefore, it needs to be converted into the frequency domain to analyze the energy distribution of the sound information in different frequency bands.
The transformation method can adopt short-time Fourier transformation, and the window of the short-time Fourier transformation is fixed, so that the frequency requirement of unsteady signal change of the electrical equipment cannot be met according to the acoustic signal characteristic analysis of the electrical equipment. Therefore, in this embodiment, a wavelet packet transform method is adopted, so that a large amount of signals of the middle-frequency and high-frequency information included in the sound information can be better analyzed in a time-frequency localization manner, and sound spectrum information can be generated.
Step 202, filtering the sound spectrum information by using a Mel filter bank, and calculating a Mel frequency cepstrum coefficient MFCC corresponding to the filtered sound spectrum information;
specifically, the mel filter bank in this embodiment employs triangular filtering, the number of the mel filter banks is set to 26, the logarithmic energy output by each filter bank is calculated, and through discrete cosine transform, the correlation between a group of output values generated by triangular filtering is removed, so as to obtain the mel-frequency cepstrum coefficient MFCC.
Step 203, adopting the first order difference and the second order difference of the Mel frequency cepstrum coefficient MFCC as the dynamic characteristics of the sound frequency spectrum information, and splicing the dynamic characteristics and the Mel frequency cepstrum coefficient MFCC to generate the sound characteristic parameters of the sound frequency spectrum information;
specifically, the obtained mel-frequency cepstrum coefficient MFCC is an acoustic feature corresponding to a short-time frame, and cannot directly reflect the dynamic change of an inter-frame acoustic signal, so that it is necessary to add a feature to represent the dynamic change between frames.
Therefore, in the present embodiment, the first-order and second-order differences of the mel-frequency cepstrum coefficients MFCC are used as the dynamic characteristics of the sound, and they are concatenated with the mel-frequency cepstrum coefficients MFCC as the sound characteristic parameters of the sound spectrum information. Wherein a first order difference dtThe calculation formula of (2) is as follows:
in the formula (d)tRepresents the t first order difference; ctRepresenting the t-th cepstral coefficient; q represents the order of the cepstral coefficient; k represents the time difference of the first derivative, and can be 1 or 2. And substituting the result in the formula again to obtain a second-order difference parameter.
Further, step 203 specifically includes:
taking the first 13 Weimeier frequency cepstrum coefficient MFCC as a characteristic parameter CtA characteristic parameter CtAnd a first order difference dtSecond order difference btThe three are respectively standardized. The normalized values are C't、d′t，b′t。
With CtFor example, the normalized calculation formula is:
respectively calculating characteristic parameters CtFirst order difference dtSecond order difference btThe information entropy E of the weight factors of the three.
It should be noted that, if a certain index (characteristic parameter C)tFirst order difference dtSecond order difference bt) The larger the entropy of the information (A) is, the smaller the degree of variation of the index value is, the smaller the amount of the information provided is, the smaller the effect of the comprehensive evaluation is, and the smaller the weight of the comprehensive evaluation is.
With the normalized characteristic parameter C'tFor example, the information entropy solving formula is as follows:
wherein the content of the first and second substances,
n represents the dimension of the characteristic parameter. In the same way, the first order difference d can be obtainedtSecond order difference btInformation entropy E ofd、Eb。
And calculating weighting factors gamma, alpha and beta through the information entropy to generate sound characteristic parameters of the sound spectrum information, wherein the calculation formula of the sound characteristic parameters is as follows:
Gt＝γCt+αdt+βbf
and step 204, extracting the characteristics of the sound information according to the sound characteristic parameters to generate a sound characteristic vector.
In the present embodiment, the method of generating the acoustic feature vector by feature extraction is not limited.
The embodiment also shows a method for fusing the defect diagnosis model of the thermal infrared video stream and the sound feature vector based on the convolutional neural network to realize the feature fusion of sound and thermal imaging. The network is named as SA-CNN, the attention module mechanism introduced into the SA-CNN realizes the functions of using convolution kernels with multiple sizes in the network, enabling the network to select proper scales and the like, and the problems of network gradient disappearance, network degradation and the like are solved by adopting an inverted residual error module mechanism.
Secondly, the network designed by the invention is light in weight in calculation, and the precision loss is controlled in a controllable range so as to greatly reduce parameters and calculated amount, compared with the conventional network, the attention module mechanism in the embodiment does not adopt standard convolution, but adopts depth separable convolution and cavity convolution, so that the parameters and calculated amount are reduced, and the network is very suitable for embedded equipment; finally, the network still has good scene applicability under the condition of no large number of customized designs, the acquisition channels are multi-source, defect diagnosis can be carried out according to single defect representation information and/or multi-defect representation information, and the network is applicable to defect diagnosis of various electric power equipment and high in applicability.
The embodiment also shows that the method for generating the defect diagnosis model specifically comprises the following steps:
and step 211, cutting the thermal infrared image subjected to frame division of the thermal infrared video stream, and performing channel number expansion to extract a first characteristic diagram of the thermal imaging image.
Specifically, the size of the framed thermal infrared video stream in the sample data set is unified to 112x112 through modes such as clipping, and then channel number expansion is performed through 1x1 convolution, batch regularization processing and activation function operation, and a first feature map of the thermal infrared video stream is extracted.
It should be noted that, in general, the activation function may be a ReLU activation function, and although the ReLU alleviates the vanishing gradient problem, the derivative of the ReLU becomes 0 when the input is negative, which may cause the problem of neuron death. Therefore, in this embodiment, the activation function is adjusted, and a calculation formula of the adjusted activation function is as follows:
in the formula, mu,
Is an adjustable parameter and μ is a value randomly drawn from a normal distribution N (0, 1) that controls when the negative part saturates.
Is the value between [1, 2), controls the fluctuation range of the negative value slope. x is an independent variable of the abscissa, and the function f (x) is a dependent variable, the output of each layer network in the network is activated, and the function has the characteristic of soft saturation when the input takes a small value, so that the robustness to noise is improved.
And 212, performing feature extraction on the first feature map by using an attention module mechanism to generate a second feature map.
In this embodiment, as shown in fig. 3, in step 212, the method for generating the second feature map specifically includes:
convolving the first feature map by using different preset convolution kernels, namely convolving the first feature map by respectively using a 3x3 depth separable convolution sum and a 3x3 void to obtain a first branch result Y1And the second branch result Y2Compared with the traditional convolution mode, the 3 × 3 depth separable convolution reduces parameters, the 3 × 3 hole convolution avoids information loss caused by pooling operation after standard convolution, the receptive field is increased while the parameter number is not increased, and the size of the receptive field is 5 × 5.
Outputting first branch results Y for different branches1And the second branch result Y2Fusing, i.e. adding element by element to obtain an intermediate feature map YcAnd two weight matrixes a and b are generated through a weighting algorithm and a full-connection layer in the convolutional network.
It should be noted that this process pools the output not only in consideration of Average Pooling (Average Pooling) but also in consideration of maximum Pooling (Max Pooling) where both average Pooling, which may feedback for each pixel of the feature map, and maximum Pooling, which may focus on local key content in the feature map, may be used to aggregate spatial information of the feature map, compared to this process in this embodiment, which focuses more on global information. Therefore, different pooling is given different weights, and the global information S on each channel is finally obtained through weighted superpositioncThe calculation formula is as follows:
in the formula, FgpFor global average pooling operation, FmpFor maximum pooling operation, H, W are respectively intermediate profiles YcC is the middle feature map YcThe number of channels of (2).
Finally, the S iscFlattening and feeding into the full connection layer FfcObtaining a vector Z and outputting two weight matrixes a and b, wherein the layer F is fully connectedfcIs two fully connected layers of firstly reducing dimension and then increasing dimension.
Performing respective Softmax activation operation on the weight matrixes a and b of the last two branches at each position to obtain respective weight of each channel in the two branches, and obtaining the weight of each channel and a first branch result Y1And the second branch result Y2Multiplying, and finally performing feature superposition to finally obtain a second feature map passing through the channel attention module.
Step 213, according to the clipped thermal infrared video stream, establishing a shortcut connection with the second feature map by using an inverse residual error module mechanism, and performing feature extraction by using an attention module mechanism to generate a third feature map;
step 214, according to the third feature map, performing feature extraction by using an attention module mechanism, and establishing a shortcut connection with the third feature map by using an inverted residual module mechanism to generate a fourth feature map;
specifically, considering that the neural network is optimized by a gradient-based BP algorithm, as parameters are propagated, a gradient diffusion problem occurs in the model, so that training is difficult to converge, and in addition, as the depth of the network is increased, the network degradation problem, namely the performance of the network is gradually increased to saturation and then rapidly decreased.
Therefore, the present embodiment introduces two inverse residual modules on the network to solve the above problems, which are: and establishing shortcut connection between the clipped thermal infrared video stream and the second feature map, and establishing shortcut connection between the third feature map after the second attention module and the feature map after the third attention module.
Step 215, performing average pooling operation on the fourth feature map, inputting the fourth feature map to a full connection layer of the convolutional neural network, and generating a thermal image feature vector;
and step 216, performing feature fusion on the thermal image feature vector and the sound feature vector by using a convolutional neural network to generate a defect diagnosis model.
Specifically, the thermal features (thermal image feature vectors) and the acoustic features are fused to flatten the obtained 13-dimensional acoustic feature parameters, the feature vectors and the thermal feature vectors are spliced, and the defect diagnosis model is generated by sequentially passing through two full-connection layers. If the number of defect types is N, the number of the neurons of the last full connecting layer is N, and classification confidence degrees are obtained through the softmax activation function, the defect diagnosis model can output a defect prediction classification result.
Further, when performing the feature extraction of the sound information, considering that the mel-frequency cepstrum coefficient MFCC is proposed based on the auditory characteristics of human ears, the recognition capability of the environmental sound of the power equipment is slightly reduced, and therefore, in combination with the sound characteristics of the operating state of the power equipment, as shown in fig. 4, the embodiment also shows a method for generating the sound spectrum information, which combines the genetic algorithm and the simulated annealing algorithm to improve the accuracy of the feature extraction of the sound information, and further improves the accuracy of the feature fusion of the sound information and the thermal infrared video stream to generate the defect diagnosis model. The method specifically comprises the following steps:
and step A, respectively adopting decimal coding for wavelet bases and decomposition layer numbers in a wavelet packet conversion mode, determining a value range, and forming double-parameter cascade codes by the two decimal codes to form an initial set, wherein each double-parameter cascade code represents a feasible solution in the initial set.
Specifically, in the embodiment, when the sound spectrum information is generated, a genetic algorithm is introduced, so as to improve the local search capability of the algorithm, facilitate intersection and variation, and save the storage space. Firstly, decimal coding is adopted for wavelet basis and decomposition layer number, the value range is determined to be [1,14], each digit code represents a gene in a genetic variation algorithm, and then two decimal codes form double-parameter cascade codes to represent a feasible solution.
In this embodiment, a random generation manner is adopted to generate 20 double-parameter concatenated codes, that is, 20 feasible solutions, to form an initial set, where each double-parameter concatenated code is a coding result in the initial set.
Step B, dividing the sound information into a plurality of data points, expressing the data points by using a sound signal sequence, calculating a sequence entropy value E (u) of the sound signal sequence according to the two-parameter cascade coding, converting the sequence entropy value E (u) into fitness, wherein the calculation formula of the fitness Fit (u) is as follows:
Fit(u)＝-E(u)
in the formula I(n,j)For the nth point data y(n,j)Signal y decomposed at j-th layerjN represents all data points of the sound signal sequence, and u is the two-parameter concatenated coding in the initial set.
It should be noted that the more uniform the signal distribution, the smaller the entropy value. The greater the fitness, the closer to the optimal solution. The smaller the sequence entropy value E (p), the greater the fitness Fit (p), the closer to the optimal solution, and the coding result (double-parameter cascade coding) with the minimum entropy value is the optimal base and the optimal decomposition level.
Therefore, the biparameter cascade coding with the minimum entropy value is found, namely the corresponding optimal wavelet packet base and the optimal decomposition layer number.
And C, calculating the fitness of each double-parameter cascade code of the initial set, randomly selecting part of double-parameter cascade codes to copy, generating a first set, and finishing copying until the scale of the new first set is the same as that of the initial set.
Specifically, a random strategy is adopted, the selected probability of each coding result is the same, 5 coding results are randomly selected to form a group each time, the fitness in each group is sequenced, the first 40 percent of coding results with high fitness are copied twice, the coding results with low fitness are eliminated at the probability of 40 percent, and the coding results with 20 percent are copied once. That is, the better the fitness value, the higher the probability of selection, the smaller the error of selection, so that the maximum probability ensures that the optimal coding result is selected and the worst coding result is eliminated.
And D, randomly pairing the coding results (double-parameter cascade codes) with the fitness higher than the average fitness, pairing the rest coding results, finally generating a plurality of completely different paired double-parameter cascade codes to generate a second set, and calculating and storing the paired fitness of each paired double-parameter cascade code in the second set, wherein the average fitness is the average value of the fitness Fit (p).
Through the method, compared with the traditional method in which any two coding results are randomly paired according to the cross probability, the pairing method in the embodiment can improve the quality of the group after cross, thereby accelerating optimization convergence.
Step E, according to the variation probability pmAnd replacing some paired double-parameter cascade codes in the second set with other double-parameter cascade codes to finally form a new coding result, namely the mutated double-parameter cascade codes to generate a third set, wherein the other double-parameter cascade codes are determined by the paired double-parameter cascade codes in the second set according to a genetic algorithm.
Wherein, in order to make the mutation probability pmThe adaptive adjustment can be carried out along with the change of the adaptability, namely when the adaptability is higher, the mutation probability becomes smaller, and when the adaptability is lower, the mutation probability becomes larger, and the mutation probability pmThe calculation formula of (2) is as follows:
in the formula, pm1、pm2To preset the probability, pm1＜2pm2In this embodiment, p is setm1＝0.08，pm2＝0.05，FmaxFor optimal coding result fitness, FmAnd the fitness of the average coding result is obtained.
In order to reserve a coding result with higher fitness (the two-parameter concatenated coding after the mutation) to the maximum extent and prevent the original coding result (the two-parameter concatenated coding in the initial set) from being damaged after the mutation, the embodiment adopts a strategy of reserving the highest fitness, that is, the difference between the fitness of the two-parameter concatenated coding after the mutation and the fitness before the mutation is calculated, if the fitness after the mutation becomes low, the coding result before the mutation is reserved, a reservation ratio is calculated as a threshold value, the two-parameter concatenated coding with lower fitness than the fitness before the mutation in the two-parameter concatenated coding after the mutation is eliminated, and the elimination rate is not greater than the threshold value.
Step F, calculating the fitness F of the optimal coding result according to the varied double-parameter cascade coding in the third setmaxFitness F with average coding resultmWhen the above condition F is not satisfiedm<Fmax<2FmThen, the solution found by the genetic algorithm at this time is considered to be the global optimal solution.
When the condition F is satisfiedm<Fmax<2FmThen, the local optimal solution is found through the genetic algorithm, and the following operations are continued: and (4) sending the first 20% of the coding results (the two-parameter cascade coding after the mutation) in the third set into a simulated annealing algorithm. Performing simulated annealing algorithm on part of the coding results, stopping iteration when the optimal fitness coding result is not changed any more, and finding out the optimal fitness coding result through the simulated annealing algorithmThe resulting two-parameter concatenated coding is a globally optimal solution.
And G, adopting the genetic algorithm and the simulated annealing algorithm in the steps C to F to respectively calculate the optimal wavelet packet basis and the optimal decomposition layer number according to the fitness so as to perform wavelet packet decomposition on the sound information subjected to the windowing processing of the subframe and generate sound frequency spectrum information.
By the method, the wavelet basis and the optimal decomposition layer number in the wavelet packet transformation mode are determined, the manual selection of the wavelet packet decomposition layer number and the wavelet basis is avoided, the characteristic extraction effect is optimized, and the accuracy of the electric power north sinking defect diagnosis is further improved.
Compared with the traditional optimization algorithm, such as a genetic algorithm and a simulated annealing algorithm, the method for generating the sound spectrum information in the embodiment solves the problems that the binary coding causes large occupied storage space, is easy to fall into a local optimal solution, and is low in convergence speed.
And 3, training the defect diagnosis model according to the sample data set to determine the operation parameters of the defect diagnosis model, wherein the trained defect diagnosis model is used for defect diagnosis of the power equipment.
The model in the embodiment can be used for acquiring data of various types of electric power equipment and training the electric power equipment, and the trained model can be deployed on a monitoring device to perform real-time reasoning analysis so as to realize defect diagnosis of different electric power equipment.
The technical scheme of the present application is described in detail above with reference to the accompanying drawings, and the present application provides a method for diagnosing defects of an electrical device based on fusion of sound source information and thermal imaging characteristics, which includes: step 1, acquiring a sample data set of the power equipment, wherein the sample data set at least comprises sound information and thermal infrared video streams which are synchronously sampled and subjected to framing processing; step 2, extracting the characteristics of the sound information, and combining the thermal infrared video stream, and performing characteristic fusion by adopting a convolutional neural network to generate a defect diagnosis model; and 3, training the defect diagnosis model according to the sample data set to determine the operation parameters of the defect diagnosis model, wherein the trained defect diagnosis model is used for defect diagnosis of the power equipment. Through the technical scheme in this application, the diagnostic advantage of full play sound and thermal imaging effectively discerns various defects of power equipment, has improved defect identification efficiency and precision, can in time discover and inform relevant maintenance personal to overhaul, ensures that power equipment is in normal operating condition to the emergence major accident.
The steps in the present application may be sequentially adjusted, combined, and subtracted according to actual requirements.
The units in the device can be merged, divided and deleted according to actual requirements.
Although the present application has been disclosed in detail with reference to the accompanying drawings, it is to be understood that such description is merely illustrative and not restrictive of the application of the present application. The scope of the present application is defined by the appended claims and may include various modifications, adaptations, and equivalents of the invention without departing from the scope and spirit of the application.
Claims (6)
1. A method for diagnosing defects of electric equipment based on fusion of sound source information and thermal imaging characteristics is characterized by comprising the following steps:
step 1, acquiring a sample data set of the power equipment, wherein the sample data set at least comprises sound information and thermal infrared video streams which are synchronously sampled and subjected to framing processing;
step 2, extracting the characteristics of the sound information, and combining the thermal infrared video stream, and performing characteristic fusion by adopting a convolutional neural network to generate a defect diagnosis model;
and 3, training the defect diagnosis model according to the sample data set to determine the operation parameters of the defect diagnosis model, wherein the trained defect diagnosis model is used for defect diagnosis of the power equipment.
2. The method for diagnosing the defect of the electric power equipment based on the fusion of the sound source information and the thermal imaging characteristics according to claim 1, wherein the step 2 of extracting the characteristics of the sound information specifically comprises the following steps:
step 201, performing frequency domain transformation on the sound information by adopting a wavelet packet transformation mode to generate sound frequency spectrum information;
step 202, filtering the sound spectrum information by using a mel filter bank, and calculating a mel-frequency cepstrum coefficient MFCC corresponding to the filtered sound spectrum information;
step 203, using the first-order and second-order differences of the Mel frequency cepstrum coefficient MFCC as the dynamic characteristics of the sound spectrum information, and splicing the dynamic characteristics and the Mel frequency cepstrum coefficient MFCC to generate sound characteristic parameters of the sound spectrum information;
and 204, performing feature extraction on the sound information according to the sound feature parameters to generate a sound feature vector.
3. The method for diagnosing defects of electric power equipment based on fusion of sound source information and thermal imaging characteristics according to claim 2, wherein the windowing is performed on the framed sound information, and the step 201 specifically includes:
respectively carrying out decimal coding on the wavelet basis and the decomposition layer number in the wavelet packet conversion mode, determining a value range, and forming double-parameter cascade coding by the two decimal codes to form an initial set;
dividing the sound information into a plurality of data points, expressing the data points by using a sound signal sequence, calculating a sequence entropy value E (u) of the sound signal sequence according to the two-parameter cascade coding, and converting the sequence entropy value E (u) into a fitness, wherein the fitness Fit (p) is calculated according to a formula as follows:
Fit(u)＝-E(u)
in the formula I(n,j)Is the nth point data point y(n,j)Signal y decomposed at j-th layerjN represents all data points of the sound signal sequence, u is the two-parameter concatenated coding in the initial set;
and according to the fitness, respectively calculating an optimal wavelet packet basis and an optimal decomposition layer number by adopting a genetic algorithm and a simulated annealing algorithm so as to perform wavelet packet decomposition on the sound information subjected to frequency division and windowing processing and generate the sound frequency spectrum information.
4. The method for diagnosing defects of electric power equipment based on fusion of sound source information and thermal imaging characteristics as claimed in claim 2, wherein the step 203 of generating the sound characteristic parameters of the sound spectrum information specifically comprises:
taking the first 13 Weimeier frequency cepstrum coefficient MFCC as a characteristic parameter CtA characteristic parameter CtAnd a first order difference dtSecond order difference btThe three are respectively standardized;
respectively calculating the characteristic parameters CtThe first order difference dtThe second order difference btThe information entropy of the weight factors of the three;
and calculating the weight factor through the information entropy to generate the sound characteristic parameter of the sound spectrum information.
5. The method for diagnosing the defect of the electrical equipment based on the fusion of the sound source information and the thermal imaging characteristics according to any one of claims 2 to 4, wherein the step 2 combines the thermal infrared video stream and performs the characteristic fusion by using a convolutional neural network to generate a defect diagnosis model, specifically comprising:
step 211, clipping the thermal infrared picture after framing the thermal infrared video stream, and performing channel number expansion to extract a first feature map of the thermal infrared picture;
step 212, utilizing an attention module mechanism to perform feature extraction on the first feature map to generate a second feature map;
step 213, according to the clipped thermal infrared video stream, establishing a shortcut connection with the second feature map by using an inverse residual error module mechanism, and performing feature extraction by using the attention module mechanism to generate a third feature map;
step 214, according to the third feature map, performing feature extraction by using the attention module mechanism, and establishing a shortcut connection with the third feature map by using the inverted residual error module mechanism to generate a fourth feature map;
step 215, performing average pooling operation on the fourth feature map, and inputting the fourth feature map to a full connection layer of the convolutional neural network to generate a thermal image feature vector;
and step 216, performing feature fusion on the thermal image feature vector and the sound feature vector by using the convolutional neural network to generate the defect diagnosis model.
6. The method according to claim 5, wherein the step 212 of generating the second feature map specifically comprises:
convolving the first characteristic diagram by using different preset convolution cores to obtain a first branch result Y1And the second branch result Y2；
The first branch result Y for different branches1The second branch result Y2Fusing, and generating two weight matrixes a and b through a weighting algorithm and a full connection layer in a convolution network;
performing Softmax activation operation on the two weight matrixes a and b of the last two branches to obtain the weight of each channel in the last two branches, and obtaining the weight of each channel and the result Y of the first branch1The second branch result Y2And multiplying to generate the second feature map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404030.0A CN112562698B (en) | 2020-12-02 | 2020-12-02 | Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404030.0A CN112562698B (en) | 2020-12-02 | 2020-12-02 | Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112562698A true CN112562698A (en) | 2021-03-26 |
| CN112562698B CN112562698B (en) | 2022-11-04 |
Family
ID=75048844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011404030.0A Active CN112562698B (en) | 2020-12-02 | 2020-12-02 | Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112562698B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114638256A (en) * | 2022-02-22 | 2022-06-17 | 合肥华威自动化有限公司 | Transformer fault detection method and system based on sound wave signals and attention network |
| CN114994437A (en) * | 2022-05-25 | 2022-09-02 | 王新华 | Fault detection method and system for power equipment |
| CN117131366A (en) * | 2023-10-26 | 2023-11-28 | 北京国电通网络技术有限公司 | Transformer maintenance equipment control method and device, electronic equipment and readable medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015101426A1 (en) * | 2013-12-31 | 2015-07-09 | Siemens Aktiengesellschaft | Devices and methods for arc fault detection |
| US20160313744A1 (en) * | 2014-04-22 | 2016-10-27 | Paul Amelio | System and method for providing and managing electricity |
| CN108875971A (en) * | 2018-06-14 | 2018-11-23 | 广东电网有限责任公司 | Device context diagnostic method, device, the diagnosis helmet and diagnosis server |
| CN110347717A (en) * | 2019-05-28 | 2019-10-18 | 国家电网有限公司 | A kind of big data storage method based on urban electric power equipment monitoring |
| JP2019216547A (en) * | 2018-06-13 | 2019-12-19 | 中部電力株式会社 | Power control apparatus, photovoltaic power generation system, and failure diagnosing method of photovoltaic power generation facility |
| CN110766059A (en) * | 2019-10-14 | 2020-02-07 | 四川西部能源股份有限公司郫县水电厂 | Transformer fault prediction method, device and equipment |
| CN111044570A (en) * | 2019-12-28 | 2020-04-21 | 广东电网有限责任公司 | Defect identification and early warning method and device for power equipment and computer equipment |
| CN111337773A (en) * | 2020-03-18 | 2020-06-26 | 江苏和光智能电气有限公司 | Transformer fault remote monitoring system and monitoring method |
| CN111523660A (en) * | 2020-04-15 | 2020-08-11 | 南京清然能源科技有限公司 | Audio-visual-thermal integrated anomaly detection and alarm method based on artificial intelligence |
| CN111562458A (en) * | 2020-06-10 | 2020-08-21 | 湖南文理学院 | Power transformer fault diagnosis method and power transformer fault diagnosis device |
| CN111784026A (en) * | 2020-05-28 | 2020-10-16 | 国网信通亿力科技有限责任公司 | Cloud-side cooperative sensing-based all-dimensional physical examination system for electrical equipment of transformer substation |
| CN111860241A (en) * | 2020-07-07 | 2020-10-30 | 国网山东省电力公司威海供电公司 | Power equipment discharge fault identification method based on wavelet packet analysis |
| CN111986172A (en) * | 2020-08-18 | 2020-11-24 | 华北电力科学研究院有限责任公司 | Infrared image fault detection method and device for power equipment |
-
2020
- 2020-12-02 CN CN202011404030.0A patent/CN112562698B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015101426A1 (en) * | 2013-12-31 | 2015-07-09 | Siemens Aktiengesellschaft | Devices and methods for arc fault detection |
| US20160313744A1 (en) * | 2014-04-22 | 2016-10-27 | Paul Amelio | System and method for providing and managing electricity |
| JP2019216547A (en) * | 2018-06-13 | 2019-12-19 | 中部電力株式会社 | Power control apparatus, photovoltaic power generation system, and failure diagnosing method of photovoltaic power generation facility |
| CN108875971A (en) * | 2018-06-14 | 2018-11-23 | 广东电网有限责任公司 | Device context diagnostic method, device, the diagnosis helmet and diagnosis server |
| CN110347717A (en) * | 2019-05-28 | 2019-10-18 | 国家电网有限公司 | A kind of big data storage method based on urban electric power equipment monitoring |
| CN110766059A (en) * | 2019-10-14 | 2020-02-07 | 四川西部能源股份有限公司郫县水电厂 | Transformer fault prediction method, device and equipment |
| CN111044570A (en) * | 2019-12-28 | 2020-04-21 | 广东电网有限责任公司 | Defect identification and early warning method and device for power equipment and computer equipment |
| CN111337773A (en) * | 2020-03-18 | 2020-06-26 | 江苏和光智能电气有限公司 | Transformer fault remote monitoring system and monitoring method |
| CN111523660A (en) * | 2020-04-15 | 2020-08-11 | 南京清然能源科技有限公司 | Audio-visual-thermal integrated anomaly detection and alarm method based on artificial intelligence |
| CN111784026A (en) * | 2020-05-28 | 2020-10-16 | 国网信通亿力科技有限责任公司 | Cloud-side cooperative sensing-based all-dimensional physical examination system for electrical equipment of transformer substation |
| CN111562458A (en) * | 2020-06-10 | 2020-08-21 | 湖南文理学院 | Power transformer fault diagnosis method and power transformer fault diagnosis device |
| CN111860241A (en) * | 2020-07-07 | 2020-10-30 | 国网山东省电力公司威海供电公司 | Power equipment discharge fault identification method based on wavelet packet analysis |
| CN111986172A (en) * | 2020-08-18 | 2020-11-24 | 华北电力科学研究院有限责任公司 | Infrared image fault detection method and device for power equipment |
Non-Patent Citations (4)
| Title |
|---|
| LI JUN ET AL: "Heating defect detection system scheme design based on infrared image processing for high voltage plant of substation", 《PROCEDIA ENGINEERING》 * |
| QIN LUO ET AL: "Temporal and spatial deep learning network for infrared thermal defect detection", 《NDT&E INTERNATIONAL》 * |
| 唐文虎等: "数据驱动的人工智能技术在电力设备状态分析中的研究与应用", 《高电压技术》 * |
| 郭铭: "基于红外成像技术的变电站电气设备热故障诊断研究", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114638256A (en) * | 2022-02-22 | 2022-06-17 | 合肥华威自动化有限公司 | Transformer fault detection method and system based on sound wave signals and attention network |
| CN114638256B (en) * | 2022-02-22 | 2024-05-31 | 合肥华威自动化有限公司 | Transformer fault detection method and system based on acoustic wave signals and attention network |
| CN114994437A (en) * | 2022-05-25 | 2022-09-02 | 王新华 | Fault detection method and system for power equipment |
| CN117131366A (en) * | 2023-10-26 | 2023-11-28 | 北京国电通网络技术有限公司 | Transformer maintenance equipment control method and device, electronic equipment and readable medium |
| CN117131366B (en) * | 2023-10-26 | 2024-02-06 | 北京国电通网络技术有限公司 | Transformer maintenance equipment control method and device, electronic equipment and readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112562698B (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112562698B (en) | Power equipment defect diagnosis method based on fusion of sound source information and thermal imaging characteristics | |
| CN110136731B (en) | Cavity causal convolution generation confrontation network end-to-end bone conduction voice blind enhancement method | |
| WO2021143327A1 (en) | Voice recognition method, device, and computer-readable storage medium | |
| CN110491416B (en) | Telephone voice emotion analysis and identification method based on LSTM and SAE | |
| CN111653289B (en) | Playback voice detection method | |
| CN111583954A (en) | Speaker independent single-channel voice separation method | |
| Guzhov et al. | Esresne (x) t-fbsp: Learning robust time-frequency transformation of audio | |
| CN112446242A (en) | Acoustic scene classification method and device and corresponding equipment | |
| CN114299995A (en) | Language emotion recognition method for emotion assessment | |
| CN111341351B (en) | Voice activity detection method, device and storage medium based on self-attention mechanism | |
| CN110363074B (en) | Humanoid recognition interaction method for complex abstract events | |
| CN116167015A (en) | Dimension emotion analysis method based on joint cross attention mechanism | |
| CN115251909B (en) | Method and device for evaluating hearing by electroencephalogram signals based on space-time convolutional neural network | |
| Eltanashi et al. | Proposed speaker recognition model using optimized feed forward neural network and hybrid time-mel speech feature | |
| Yang et al. | Research on evaluation model for vehicle interior sound quality based on an optimized BiLSTM using genetic algorithm | |
| CN114283301A (en) | Self-adaptive medical image classification method and system based on Transformer | |
| CN114863939B (en) | Panda attribute identification method and system based on sound | |
| Shen et al. | Home activity monitoring based on gated convolutional neural networks and system fusion | |
| Yang et al. | Audio scene classification based on gated recurrent unit | |
| CN117524252B (en) | Light-weight acoustic scene perception method based on drunken model | |
| CN114496221B (en) | Automatic depression diagnosis system based on closed-loop voice chain and deep learning | |
| Zhang | Dynamic Feature Extraction Method of Phone Speakers Based on Deep Learning | |
| CN111833885B (en) | Sound source identification method based on convolution fuzzy neural network | |
| CN117711443A (en) | Lightweight speech emotion recognition method and system based on multi-scale attention | |
| CN116453548A (en) | Voice emotion recognition method based on attention MCNN combined with gender information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |