CN114495983A - Equipment failure voiceprint monitoring system based on cloud edge collaboration - Google Patents
Equipment failure voiceprint monitoring system based on cloud edge collaboration Download PDFInfo
- Publication number
- CN114495983A CN114495983A CN202210158492.1A CN202210158492A CN114495983A CN 114495983 A CN114495983 A CN 114495983A CN 202210158492 A CN202210158492 A CN 202210158492A CN 114495983 A CN114495983 A CN 114495983A
- Authority
- CN
- China
- Prior art keywords
- voiceprint
- fault
- data
- cloud
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 48
- 230000002159 abnormal effect Effects 0.000 claims abstract description 59
- 239000013598 vector Substances 0.000 claims abstract description 51
- 238000004458 analytical method Methods 0.000 claims abstract description 40
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 37
- 238000012795 verification Methods 0.000 claims description 20
- 238000007906 compression Methods 0.000 claims description 8
- 230000006835 compression Effects 0.000 claims description 7
- 230000035945 sensitivity Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 12
- 230000005856 abnormality Effects 0.000 abstract description 9
- 238000013135 deep learning Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 14
- 210000002569 neuron Anatomy 0.000 description 12
- 238000001514 detection method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 210000000988 bone and bone Anatomy 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013499 data model Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000007670 refining Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 238000004804 winding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention discloses an equipment failure voiceprint monitoring system based on cloud edge coordination, which comprises a cloud sample library and a template library, wherein the cloud sample library and the template library are established for equipment voiceprint failure data; acquiring equipment audio data, performing preprocessing operation, and extracting first audio voiceprint features; determining the abnormality of the voiceprint by comparing the first high-dimensional representation data corresponding to the first audio voiceprint feature with the high-dimensional representation data of the normal voiceprint; extracting voiceprint characteristics corresponding to the fault audio data; and comparing the feature vector of the voiceprint feature corresponding to the audio data of the equipment with a voiceprint feature fault template in a template library to determine the corresponding fault type. The method can deploy the equipment fault voiceprint monitoring deep learning algorithm and continuously optimize the accuracy of the algorithm under the condition that a former marked sample library is rare, so that the difficulty of fault data acquisition is solved, and the identification accuracy is improved; and only the uploaded abnormal voiceprints need to be analyzed based on the side fault identification and analysis module, so that the server overhead is greatly reduced.
Description
Technical Field
The invention relates to a fault voiceprint monitoring technology, in particular to an equipment fault voiceprint monitoring system based on cloud edge coordination.
Background
During the operation of the electric equipment, vibration occurs between the machine body and the firmware, the parts or between the parts themselves and mechanical waves are generated, so that the equipment makes a sound, which is called as language expression of the equipment (hereinafter referred to as voiceprint). The voiceprint changes along with the change of the running state of the equipment, and particularly, after the equipment is in a defect or a fault, the language of an internal component or a structure is changed due to the mechanical deformation of the internal component or the structure, so that the voiceprint contains a large amount of equipment state information and can be used as an important characteristic parameter for diagnosing the equipment and the fault. On the other hand, compared with many traditional detection and identification methods, the voiceprint identification method based on the vibro-acoustic signals can realize the detection of the running state of the device without power outage outside the device, has no electrical connection with electrical equipment, and does not influence the normal running of the system. The equipment detection early warning based on the vibration sound signals can timely master the running state of the equipment and the change of the surrounding environment, discover the defects of the equipment and the hidden danger endangering the safety, quickly take effective measures and ensure the safety and the system stability of the power equipment.
The power industry has the characteristics of long chain, multiple services and the like, and the lack of data standardization, data content, data annotation and the like is serious. The prior art has the technical defect of insufficient value mining of the voiceprint data of the equipment in the identification mode of the voiceprint of the equipment, the prior art basically adopts a manual mode to slowly accumulate original data, the efficiency is low, the collection standard specification is not uniform, and a lot of data are difficult to obtain and collect, especially fault and abnormal data (such as partial discharge, winding deformation) and data under a complex appearance environment, the fault data are incomplete and non-uniform in sequence, so that the identification accuracy is low; and the data size of the identified data template is extremely large, while the prior art is usually based on a unilateral identification mode of an edge end, so that the system overhead is extremely large.
Disclosure of Invention
The invention mainly aims to provide an equipment failure voiceprint monitoring system based on cloud edge coordination, and aims to solve the problems that in the prior art, equipment failure voiceprint recognition system cost is extremely high, failure and abnormal data and data acquisition difficulty under a complex appearance environment are high, and system analysis and recognition capacity is improved.
In order to achieve the above object, an apparatus failure voiceprint monitoring system based on cloud edge coordination includes:
the cloud database module is used for establishing a device voiceprint fault data cloud sample database and a template database;
the acquisition module is used for acquiring the audio data of the equipment, carrying out preprocessing operation and extracting first audio voiceprint characteristics;
the voiceprint anomaly monitoring module is used for acquiring first high-dimensional representation data corresponding to the first audio voiceprint characteristics based on a coding network, and comparing the first high-dimensional representation data corresponding to the first audio voiceprint characteristics with the high-dimensional representation data of normal voiceprints to obtain specificity scores of the first high-dimensional representation data and the normal voiceprints; if the specificity score exceeds a set abnormal monitoring threshold value, determining the audio data as fault audio data;
the side end fault identification and analysis module is connected with the cloud base module and the sound pattern abnormity monitoring module; extracting voiceprint characteristics corresponding to the fault audio data; and comparing the feature vector of the voiceprint feature corresponding to the audio data of the equipment with a voiceprint feature fault template in a template library to determine the corresponding fault type.
Preferably, the cloud library module registers a device voiceprint characteristic fault template in advance to form a template library;
issuing a fault model from the cloud template library to the side fault identification and analysis module;
the side end fault identification and analysis module further comprises: obtaining a similarity score of each fault type by comparing the feature vector of the voiceprint features corresponding to the audio data of the equipment with the voiceprint feature fault template corresponding to each fault type in the template library, and taking the fault type corresponding to the highest similarity score as the equipment fault type to obtain an identification result;
and the side end fault identification and analysis module uploads the identification result and sends fault information and a sample to a cloud sample library.
Preferably, the voiceprint anomaly monitoring module further comprises a network training sub-module, and the network training sub-step comprises:
inputting the voiceprint characteristics of normal audio through an input device; the coding network codes the voiceprint characteristics into high-dimensional representation through a neural network; the decoding network decodes the high-dimensional features into voiceprint features of the audio, and the voiceprint features before encoding and the decoded voiceprint features are compared to train a network structure; and storing the trained network structure.
Preferably, the edge fault identification and analysis module further includes a model training sub-module, and the model training sub-module includes:
in the training stage, the input data is a K-dimensional key frequency vector set V after frequency compression of voiceprint datainput(ii) a Output layer E8The output data is set VoutputThe loss function for each batch of data used in the inverse transfer of the training gradient is expressed as follows:
where N is the number of data per batch, vinput∈Vinput,voutput∈VoutputIn which α iskIs a dimension loss weight, beta is a loss sensitivity coefficient, where alpha iskSuppressing the low-dimensional weight and increasing the high-dimensional weight for a finite nonlinear equalization functionA logarithm-based function is generally used:
αk＝Alog(k)+B
wherein A and B are compensation coefficients, and are adjusted according to specific precision and sensitivity requirements; the output layer C outputs a prediction difference degree s, the expected difference degree under the self-supervision condition is 0, and a cross entropy loss function is used for training;
the side end fault identification analysis module inputs K-dimensional key frequency vector set V with compressed voiceprint data frequency in a verification stageinputAnd outputting the difference degree of the output layer C as the abnormal monitoring result of the voiceprint data.
Preferably, the cloud base module further comprises an algorithm updating submodule; the algorithm updating submodule comprises:
the fault audio data are uploaded to a template base and a sample base of the cloud end to expand a database; issuing a fault model from the cloud template library to the side fault identification and analysis module, and performing algorithm iterative optimization by updating a template;
collecting abnormal frequency characteristic vector VabnDivided into 50Hz frequency multiplication vectors and other sets of frequency vectors, denoted as
And
manually marking the abnormity positioned by the voiceprint data and arranging the abnormity into a standard abnormity marking file;
will be provided with
And
merging the abnormal standard files into an abnormal voiceprint compression frequency characteristic standard data set;
in the abnormal voiceprint registration stage, respectively registering an abnormal voiceprint compressed frequency feature standard data set as an abnormal voiceprint intrinsic frequency feature dictionary and an abnormal voiceprint extrinsic frequency feature dictionary according to two different sets;
and the side end fault identification analysis module inputs verification frequency characteristic vectors in a verification stage, divides the verification frequency characteristic vectors into 50Hz frequency multiplication characteristic vectors and other frequency characteristic vectors, and respectively compares the verification frequency characteristic vectors with the abnormal voiceprint eigenfrequency characteristic dictionary and the abnormal voiceprint extrinsic frequency characteristic dictionary by a template matching algorithm to obtain the intrinsic abnormal and extrinsic abnormal categories which are most similar to the current verification characteristic vectors.
The device failure voiceprint monitoring system based on cloud edge collaboration provided by the invention is characterized in that a device voiceprint failure data cloud sample library and a template library are established; acquiring equipment audio data, performing preprocessing operation, and extracting first audio voiceprint features; acquiring first high-dimensional representation data corresponding to the first audio voiceprint feature based on a coding network, and comparing the first high-dimensional representation data corresponding to the first audio voiceprint feature with the high-dimensional representation data of a normal voiceprint to obtain specificity scores of the first high-dimensional representation data and the normal voiceprint; if the specificity score exceeds a set abnormal monitoring threshold value, determining the audio data as fault audio data; the side end fault identification and analysis module is connected with the cloud base module and the sound pattern abnormity monitoring module; extracting voiceprint characteristics corresponding to the fault audio data; and comparing the feature vector of the voiceprint feature corresponding to the audio data of the equipment with a voiceprint feature fault template in a template library to determine the corresponding fault type. The method does not depend on the existing fault sample library, can deploy the equipment fault voiceprint monitoring deep learning algorithm and continuously optimize the accuracy of the algorithm under the condition that the early-stage marked sample library is rare, thereby solving the difficulty of fault data acquisition and improving the identification accuracy; and only the uploaded abnormal voiceprints need to be analyzed by the side-end-based fault identification and analysis module, so that the server overhead is greatly reduced, and the algorithm of the server is optimized through a cloud-side cooperation mechanism.
Drawings
FIG. 1 is a schematic structural diagram of an apparatus failure voiceprint monitoring system based on cloud edge coordination according to the present invention;
FIG. 2 is a schematic structural diagram of a voiceprint anomaly monitoring module according to the present invention;
FIG. 3 is a schematic structural diagram of an edge fault identification and analysis module according to the present invention;
FIG. 4 is a schematic diagram of the operation of the system of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings of the specification, it being understood that the preferred embodiments described herein are merely for illustrating and explaining the present invention, and are not intended to limit the present invention, and that the embodiments and features of the embodiments in the present invention may be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a first embodiment of an equipment failure voiceprint monitoring system based on cloud edge coordination, in this embodiment, the equipment failure voiceprint monitoring system based on cloud edge coordination includes:
the cloud database module is used for establishing a device voiceprint fault data cloud sample database and a template database;
the cloud sample library collects voiceprint data collected by the station end, can flexibly manage and display the voiceprint data according to equipment types, voltage levels, abnormal types, occurrence time and the like, and has the functions of sample statistics, sample classification and the like; the cloud template base collects fault templates, the fault samples are learned into the fault templates through manual marking or algorithms and stored in the template base, and meanwhile the cloud template base has a fault template issuing function.
Specifically, in this embodiment, the cloud sample library is used for collecting voiceprint data collected by the station side, storing and performing classified management, and has functions of sample statistics and sample classified downloading; the cloud template base is used for collecting fault templates, the modules have an iterative optimization function, fault samples are learned into fault templates through manual marking or an algorithm and stored in the template base, the template base can be synchronously expanded along with expansion of the database, the fault template feature base is refined, and meanwhile, the module has a fault template issuing function.
The acquisition module acquires the audio data of the equipment, performs preprocessing operation and extracts the first audio voiceprint feature.
In particular, audio data is collected by means of an ultrasound conducting device, a bone conducting device and an air conducting device.
As shown in fig. 2, in this embodiment, the voiceprint abnormality monitoring module obtains first high-dimensional representation data corresponding to the first audio voiceprint feature based on a coding network, and compares the first high-dimensional representation data corresponding to the first audio voiceprint feature with high-dimensional representation data of a normal voiceprint to obtain specificity scores of the first high-dimensional representation data and the normal voiceprint; if the specificity score exceeds a set abnormal monitoring threshold value, determining the audio data as fault audio data;
specifically, the voiceprint anomaly monitoring module collects audio files uploaded by the audio acquisition equipment, performs real-time analysis on the presence or absence of anomalies, uploads the analyzed anomalous voiceprint files to the side end, and supports the self-learning function of the equipment.
Preferably, the cloud library module registers a device voiceprint characteristic fault template in advance to form a template library;
issuing a fault model from the cloud template library to the side fault identification and analysis module;
the side end fault identification and analysis module is connected with the cloud base module and the sound pattern abnormity monitoring module; extracting voiceprint characteristics corresponding to the fault audio data; and comparing the feature vector of the voiceprint feature corresponding to the audio data of the equipment with a voiceprint feature fault template in a template library to determine the corresponding fault type.
Specifically, as shown in fig. 3, which is a schematic diagram of the edge fault identification and analysis module of the present invention, in this embodiment, the edge fault identification and analysis module is connected to the cloud sample library, the template library module and the edge voiceprint abnormality monitoring module, and can receive real-time abnormal voiceprint data uploaded by the voiceprint abnormality monitoring module and a fault template issued by the cloud, and implement voiceprint identification of equipment faults through a fault template matching algorithm, and upload fault information and a sample to the cloud sample library, and the edge fault identification and analysis module only needs to analyze the uploaded abnormal voiceprint, thereby greatly reducing server overhead.
And issuing a fault data model to an edge fault recognition analysis module in the cloud template library, iteratively training a deep learning model, optimizing a voiceprint fault recognition algorithm, and refining the influence of different parameters on the algorithm accuracy.
Preferably, the voiceprint anomaly monitoring module further comprises a network training sub-module, and the network training sub-step comprises: inputting the voiceprint characteristics of normal audio through an input device; the coding network codes the voiceprint characteristics into high-dimensional representation through a neural network; the decoding network decodes the high-dimensional features into voiceprint features of the audio, and the voiceprint features before encoding and the decoded voiceprint features are compared to train a network structure; and storing the trained network structure.
Specifically, in this embodiment, taking a transformer as an example, a self-monitoring multilayer neural network model may be designed for a compression frequency characteristic of the transformer, and a working voiceprint characteristic of the transformer may be learned by self-monitoring, and differences between voiceprints may be determined, specifically including the following steps:
constructing a multilayer fully-connected neural network model, wherein each layer is sequentially represented as [ E ]1,E2,E3,E4,E5,E6,E7,E8]The specific parameters are as follows:
input layer E1The method comprises K neurons, wherein K is a feature vector dimension, and 20% of the neurons are randomly abandoned for training in a training stage;
hidden layer E2300 neurons are included, and 20% of neurons are abandoned randomly for training in the training stage;
hidden layer E3The training stage randomly abandons 10% of neurons for training;
hidden layer E4The training stage randomly abandons 10% of neurons for training;
hidden layer E5The training stage randomly abandons 10% of neurons for training;
hidden layer E6The training stage randomly abandons 10% of neurons for training;
hidden layer E7300 neurons are included, and 20% of neurons are abandoned randomly for training in the training stage;
output layer E8Contains 300 neurons, the output is K-dimensional eigenvector dimensions.
The output layer C is connected to the hidden layer E4Then, 32 neurons were included, and the output was 1-dimensional difference.
An application stage of the anomaly monitoring module: importing real-time audio voiceprint features of the equipment, acquiring high-dimensional representation of the voiceprint features through a coding network and comparing the high-dimensional representation with that of a normal voiceprint; and calculating specificity scores of the two, wherein the scores exceed a set abnormality monitoring threshold value to trigger an abnormality alarm.
Further, in the monitoring task, when continuous voiceprint data characteristics are input, the neural network model outputs a continuous voiceprint abnormal monitoring difference degree result S [ [ S ] ]1,s2,...,sN]. In order to obtain an adaptive detection alarm, the following steps are carried out:
step 1: setting an alarm trigger discrepancy threshold tausIf the threshold value is not exceeded, the current abnormity is considered to be not abnormal;
step 2: setting an alarm trigger frequency threshold taufWhen the alarm trigger frequency exceeds the threshold value in a period of time T, the difference threshold value is automatically increased to make the trigger frequency and the trigger threshold value have negative correlation, which is expressed as,
and step 3: setting the maximum value of alarm trigger difference
As the maximum value for triggering the adjustment of the difference degree, if the threshold value of the difference degree reaches the maximum value, even if the triggering frequency exceeds the threshold value, the self-adaptive adjustment is not carried out any more;
and 4, step 4: through self-adaptive adjustment, the monitoring algorithm outputs a self-adaptive difference degree result
And 5: according to the self-adaptive difference result, obtaining the frame position corresponding to the abnormal alarm, and according to the input key frequency vector set VinputObtaining a set V of abnormal frequency characteristic vectorsabnAnd realizing abnormal voiceprint time positioning and feature acquisition.
In this embodiment, the frontier fault identification and analysis module further includes: obtaining a similarity score of each fault type by comparing the feature vector of the voiceprint features corresponding to the audio data of the equipment with the voiceprint feature fault template corresponding to each fault type in the template library, and taking the fault type corresponding to the highest similarity score as the equipment fault type to obtain an identification result;
specifically, in this embodiment, the determination of the fault type is implemented by a template matching method: the device compares the real-time feature vector with a voiceprint feature fault template in a template library to obtain the highest similarity score output by each fault type, the results are arranged from high to low according to the similarity of each fault, the default comparison method is cosine similarity, and the higher the cosine similarity value is, the higher the similarity is.
Preferably, the side fault identification and analysis module sends the identification result and sends the fault information and the sample to the cloud sample library.
Specifically, as shown in fig. 4, which is a schematic diagram of an operation flow of the system according to this embodiment, the audio data may be collected by an ultrasonic conduction device, a bone conduction device, and an air conduction device, and the audio data is processed by a convergence layer and then uploaded to the voiceprint abnormality detection system to be processed to obtain abnormal audio data; the cloud sample library module collects voiceprint data collected by the station end and has the functions of sample statistics, sample classification and the like; the cloud template library module collects fault templates, learns the fault samples into fault templates through manual marking or an algorithm and stores the fault templates in the template library; the voiceprint anomaly detection module collects the audio files uploaded by the audio acquisition equipment, analyzes whether the voiceprint files exist in real time or not, and uploads the analyzed abnormal voiceprint files to a side end; the edge fault identification and analysis module is connected with the cloud sample library, the template library module and the edge voiceprint abnormity monitoring module, can receive real-time abnormal voiceprint data uploaded by the voiceprint abnormity monitoring module and a fault template issued by the cloud, realizes voiceprint identification of equipment faults through a fault template matching algorithm and uploads fault information and samples to the cloud sample library, and the edge fault identification and analysis module only needs to analyze the uploaded abnormal voiceprints; and issuing a fault data model to an edge fault recognition analysis module in the cloud template library, iteratively training a deep learning model, optimizing a voiceprint fault recognition algorithm, and refining the influence of different parameters on the algorithm accuracy.
Preferably, the edge fault identification and analysis module further includes a model training sub-module, and the model training sub-module includes:
in the training stage, the input data is a K-dimensional key frequency vector set V after frequency compression of voiceprint datainput(ii) a Output layer E8The output data is set VoutputThe loss function for each batch of data used in the inverse transfer of the training gradient is expressed as follows:
where N is the number of data per batch, vinput∈Vinput,voutput∈VoutputIn which α iskIs a dimension loss weight, beta is a loss sensitivity coefficient, where alpha iskFor a finite nonlinear equalization function, suppressing low-dimensional weights and increasing high-dimensional weights, a logarithm-based function is generally used:
αk＝Alog(k)+B
wherein A and B are compensation coefficients, and are adjusted according to specific precision and sensitivity requirements; the output layer C outputs a prediction difference degree s, the expected difference degree under the self-supervision condition is 0, and a cross entropy loss function is used for training;
the edgeIn the verification stage of the end fault identification and analysis module, input data are K-dimensional key frequency vector set V after voiceprint data frequency compressioninputAnd outputting the difference degree of the output layer C as the abnormal monitoring result of the voiceprint data.
Preferably, the cloud base module further comprises an algorithm updating submodule; the algorithm updating submodule comprises:
the fault audio data are uploaded to a template base and a sample base of the cloud end to expand a database; issuing a fault model from the cloud template library to the side fault identification and analysis module, and performing algorithm iterative optimization by updating a template;
specifically, in this embodiment, the abnormal audio data is uploaded to a fault identification and analysis module for analysis, so as to obtain fault audio data; uploading the fault audio data to a template base and a sample base of the cloud to expand a database; and issuing a fault model from the cloud template library to the fault identification and analysis module, and performing algorithm iterative optimization by updating a template so as to further improve the identification capability of the analysis module.
The method comprises the following steps of identifying and detecting abnormal frequency characteristic vectors by using a template matching algorithm, and realizing generalization detection by separating and registering abnormal frequency characteristics, wherein the method comprises the following specific steps:
set V of abnormal frequency characteristic vectorabnDivided into 50Hz frequency multiplication vectors and other sets of frequency vectors, denoted as
And
manually marking the abnormity positioned by the voiceprint data and arranging the abnormity into a standard abnormity marking file;
will be provided with
And
merging the abnormal standard files into an abnormal voiceprint compression frequency characteristic standard data set;
in the abnormal voiceprint registration stage, respectively registering an abnormal voiceprint compressed frequency feature standard data set as an abnormal voiceprint intrinsic frequency feature dictionary and an abnormal voiceprint extrinsic frequency feature dictionary according to two different sets;
in this embodiment, the side end fault identification and analysis module inputs a verification frequency feature vector in a verification stage, divides the verification frequency feature vector into a 50Hz frequency multiplication feature vector and other frequency feature vectors, and performs template matching algorithm comparison with the abnormal voiceprint eigenfrequency feature dictionary and the abnormal voiceprint extrinsic frequency feature dictionary respectively to obtain an intrinsic abnormality and an extrinsic abnormality category most similar to the current verification feature vector.
Specifically, in the verification stage, the verification frequency feature vector is input, and is divided into a 50Hz frequency multiplication feature vector and other frequency feature vectors, and the frequency multiplication feature vector and the other frequency feature vectors are respectively compared with the abnormal voiceprint eigenfrequency feature dictionary and the abnormal voiceprint extrinsic frequency feature dictionary by the template matching algorithm. Such as a dynamic time warping method, a cosine similarity method, a gaussian mixture model method, etc. may be used. And finally obtaining the intrinsic anomaly and extrinsic anomaly types most similar to the current verification feature vector.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (5)
1. The utility model provides an equipment trouble voiceprint monitoring system based on cloud limit is cooperative which characterized in that includes:
the cloud database module is used for establishing a device voiceprint fault data cloud sample database and a template database;
the acquisition module is used for acquiring the audio data of the equipment, carrying out preprocessing operation and extracting first audio voiceprint characteristics;
the voiceprint anomaly monitoring module is used for acquiring first high-dimensional representation data corresponding to the first audio voiceprint characteristics based on a coding network, and comparing the first high-dimensional representation data corresponding to the first audio voiceprint characteristics with the high-dimensional representation data of normal voiceprints to obtain specificity scores of the first high-dimensional representation data and the normal voiceprints; if the specificity score exceeds a set abnormal monitoring threshold value, determining the audio data as fault audio data;
the side end fault identification and analysis module is connected with the cloud base module and the sound pattern abnormity monitoring module; extracting voiceprint characteristics corresponding to the fault audio data; and comparing the feature vector of the voiceprint feature corresponding to the audio data of the equipment with a voiceprint feature fault template in a template library to determine the corresponding fault type.
2. The system for monitoring the voiceprint of the equipment fault based on the cloud edge collaboration as claimed in claim 1, wherein the cloud library module registers an equipment voiceprint characteristic fault template in advance to form a template library;
issuing a fault model from the cloud template library to the side fault identification and analysis module;
the side end fault identification and analysis module further comprises: obtaining a similarity score of each fault type by comparing the feature vector of the voiceprint features corresponding to the audio data of the equipment with the voiceprint feature fault template corresponding to each fault type in the template library, and taking the fault type corresponding to the highest similarity score as the equipment fault type to obtain an identification result;
and the side end fault identification and analysis module uploads the identification result and sends fault information and a sample to a cloud sample library.
3. The cloud-edge-collaboration-based equipment failure voiceprint monitoring system according to any one of claims 1-2, wherein the voiceprint anomaly monitoring module further comprises a network training sub-module, and the network training sub-step comprises:
inputting the voiceprint characteristics of normal audio through an input device; the coding network codes the voiceprint characteristics into high-dimensional representation through a neural network; the decoding network decodes the high-dimensional features into voiceprint features of the audio, and the voiceprint features before encoding and the decoded voiceprint features are compared to train a network structure; and storing the trained network structure.
4. The cloud-edge-collaboration-based equipment failure voiceprint monitoring system of claim 2 wherein the edge-side failure recognition analysis module further comprises a model training sub-module, the model training sub-module comprising:
in the training stage, the input data is a K-dimensional key frequency vector set V after frequency compression of voiceprint datainput(ii) a Output layer E8The output data is set VoutputThe loss function for each batch of data used in the inverse transfer of the training gradient is expressed as follows:
where N is the number of data per batch, vinput∈Vinput,voutput∈VoutputIn which α iskIs a dimension loss weight, beta is a loss sensitivity coefficient, where alpha iskFor a finite nonlinear equalization function, suppressing low-dimensional weights and increasing high-dimensional weights, a logarithm-based function is used:
αk＝A log(k)+B
wherein A and B are compensation coefficients, and are adjusted according to specific precision and sensitivity requirements; the output layer C outputs a prediction difference degree s, the expected difference degree under the self-supervision condition is 0, and a cross entropy loss function is used for training;
the side end fault identification analysis module inputs K-dimensional key frequency vector set V with compressed voiceprint data frequency in a verification stageinputAnd outputting the difference degree of the output layer C as the abnormal monitoring result of the voiceprint data.
5. The cloud-edge-collaboration-based equipment failure voiceprint monitoring system of claim 1, wherein the cloud library module further comprises an algorithm update sub-module; the algorithm updating submodule comprises:
the fault audio data are uploaded to a template base and a sample base of the cloud end to expand a database; issuing a fault model from the cloud template library to the side fault identification and analysis module, and performing algorithm iterative optimization by updating a template;
collecting abnormal frequency characteristic vector VabnDivided into 50Hz frequency multiplication vectors and other sets of frequency vectors, denoted as
And
manually marking the abnormity positioned by the voiceprint data and arranging the abnormity into a standard abnormity marking file;
will be provided with
And
merging the abnormal standard files into an abnormal voiceprint compression frequency characteristic standard data set;
in the abnormal voiceprint registration stage, respectively registering an abnormal voiceprint compressed frequency feature standard data set as an abnormal voiceprint intrinsic frequency feature dictionary and an abnormal voiceprint extrinsic frequency feature dictionary according to two different sets;
and the side end fault identification analysis module inputs verification frequency characteristic vectors in a verification stage, divides the verification frequency characteristic vectors into 50Hz frequency multiplication characteristic vectors and other frequency characteristic vectors, and respectively compares the verification frequency characteristic vectors with the abnormal voiceprint eigenfrequency characteristic dictionary and the abnormal voiceprint extrinsic frequency characteristic dictionary by a template matching algorithm to obtain the intrinsic abnormal and extrinsic abnormal categories which are most similar to the current verification characteristic vectors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158492.1A CN114495983B (en) | 2022-02-21 | 2022-02-21 | Equipment fault voiceprint monitoring system based on cloud edge cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158492.1A CN114495983B (en) | 2022-02-21 | 2022-02-21 | Equipment fault voiceprint monitoring system based on cloud edge cooperation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114495983A true CN114495983A (en) | 2022-05-13 |
| CN114495983B CN114495983B (en) | 2024-10-29 |
Family
ID=81482334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210158492.1A Active CN114495983B (en) | 2022-02-21 | 2022-02-21 | Equipment fault voiceprint monitoring system based on cloud edge cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114495983B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115035913A (en) * | 2022-08-11 | 2022-09-09 | 合肥中科类脑智能技术有限公司 | Sound abnormity detection method |
| CN115588439A (en) * | 2022-12-13 | 2023-01-10 | 杭州兆华电子股份有限公司 | Fault detection method and device of voiceprint acquisition device based on deep learning |
| CN116774109A (en) * | 2023-06-26 | 2023-09-19 | 国网黑龙江省电力有限公司佳木斯供电公司 | Transformer fault identification system based on voiceprint detection information |
| CN117914003A (en) * | 2024-03-19 | 2024-04-19 | 沈阳智帮电气设备有限公司 | Intelligent monitoring auxiliary method and system for box-type transformer based on cloud edge cooperation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105424395A (en) * | 2015-12-15 | 2016-03-23 | 珠海格力电器股份有限公司 | Method and device for determining equipment fault |
| US20190166442A1 (en) * | 2017-11-29 | 2019-05-30 | Fujitsu Limited | Abnormality detecting device, abnormality detection method, and abnormality detection computer program |
| CN110309692A (en) * | 2018-03-27 | 2019-10-08 | 杭州海康威视数字技术股份有限公司 | Face identification method, apparatus and system, model training method and device |
| US20200233397A1 (en) * | 2019-01-23 | 2020-07-23 | New York University | System, method and computer-accessible medium for machine condition monitoring |
| CN111707355A (en) * | 2020-06-19 | 2020-09-25 | 浙江讯飞智能科技有限公司 | Equipment operation state detection method, device, equipment and storage medium |
| CN112735438A (en) * | 2020-12-29 | 2021-04-30 | 科大讯飞股份有限公司 | Online voiceprint feature updating method and device, storage device and modeling device |
| CN112885372A (en) * | 2021-01-15 | 2021-06-01 | 国网山东省电力公司威海供电公司 | Intelligent diagnosis method, system, terminal and medium for power equipment fault sound |
| US20210272580A1 (en) * | 2020-03-02 | 2021-09-02 | Espressif Systems (Shanghai) Co., Ltd. | System and method for offline embedded abnormal sound fault detection |
-
2022
- 2022-02-21 CN CN202210158492.1A patent/CN114495983B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105424395A (en) * | 2015-12-15 | 2016-03-23 | 珠海格力电器股份有限公司 | Method and device for determining equipment fault |
| US20190166442A1 (en) * | 2017-11-29 | 2019-05-30 | Fujitsu Limited | Abnormality detecting device, abnormality detection method, and abnormality detection computer program |
| CN110309692A (en) * | 2018-03-27 | 2019-10-08 | 杭州海康威视数字技术股份有限公司 | Face identification method, apparatus and system, model training method and device |
| US20200233397A1 (en) * | 2019-01-23 | 2020-07-23 | New York University | System, method and computer-accessible medium for machine condition monitoring |
| US20210272580A1 (en) * | 2020-03-02 | 2021-09-02 | Espressif Systems (Shanghai) Co., Ltd. | System and method for offline embedded abnormal sound fault detection |
| CN111707355A (en) * | 2020-06-19 | 2020-09-25 | 浙江讯飞智能科技有限公司 | Equipment operation state detection method, device, equipment and storage medium |
| CN112735438A (en) * | 2020-12-29 | 2021-04-30 | 科大讯飞股份有限公司 | Online voiceprint feature updating method and device, storage device and modeling device |
| CN112885372A (en) * | 2021-01-15 | 2021-06-01 | 国网山东省电力公司威海供电公司 | Intelligent diagnosis method, system, terminal and medium for power equipment fault sound |
Non-Patent Citations (1)
| Title |
|---|
| 吴帆;刘艳霞;刘力铭;何彦德;: "基于深度学习模型的电力变压器故障声音诊断方法研究", 电声技术, no. 01, 5 January 2020 (2020-01-05) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115035913A (en) * | 2022-08-11 | 2022-09-09 | 合肥中科类脑智能技术有限公司 | Sound abnormity detection method |
| CN115588439A (en) * | 2022-12-13 | 2023-01-10 | 杭州兆华电子股份有限公司 | Fault detection method and device of voiceprint acquisition device based on deep learning |
| CN116774109A (en) * | 2023-06-26 | 2023-09-19 | 国网黑龙江省电力有限公司佳木斯供电公司 | Transformer fault identification system based on voiceprint detection information |
| CN116774109B (en) * | 2023-06-26 | 2024-01-30 | 国网黑龙江省电力有限公司佳木斯供电公司 | Transformer fault identification system based on voiceprint detection information |
| CN117914003A (en) * | 2024-03-19 | 2024-04-19 | 沈阳智帮电气设备有限公司 | Intelligent monitoring auxiliary method and system for box-type transformer based on cloud edge cooperation |
| CN117914003B (en) * | 2024-03-19 | 2024-05-24 | 沈阳智帮电气设备有限公司 | Intelligent monitoring auxiliary method and system for box-type transformer based on cloud edge cooperation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114495983B (en) | 2024-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114495983B (en) | Equipment fault voiceprint monitoring system based on cloud edge cooperation | |
| CN111259921B (en) | Transformer sound abnormality detection method based on improved wavelet packet and deep learning | |
| CN110890102A (en) | Engine defect detection algorithm based on RNN voiceprint recognition | |
| Mechefske et al. | Fault detection and diagnosis in low speed rolling element bearings Part II: The use of nearest neighbour classification | |
| CN115409131B (en) | Production line abnormity detection method based on SPC process control system | |
| CN108304567B (en) | Method and system for identifying working condition mode and classifying data of high-voltage transformer | |
| CN110261116A (en) | A kind of Bearing Fault Detection Method and device | |
| CN115424635B (en) | Cement plant equipment fault diagnosis method based on sound characteristics | |
| CN113792597A (en) | Mechanical equipment abnormal sound detection method based on self-supervision feature extraction | |
| CN116861303A (en) | Digital twin multisource information fusion diagnosis method for transformer substation | |
| CN116735170A (en) | Intelligent fault diagnosis method based on self-attention multi-scale feature extraction | |
| CN117591857A (en) | Motor detection method based on deep learning | |
| CN117437933A (en) | Feature cluster combination generation type learning-based unsupervised detection method for fault of voiceprint signal of transformer | |
| CN115457980A (en) | Automatic voice quality evaluation method and system without reference voice | |
| KR102366787B1 (en) | Real-time sliding window based anomaly detection system for multivariate data generated by manufacturing equipment | |
| CN113255591A (en) | Bearing fault diagnosis method based on random forest and fusion characteristics | |
| CN113450827A (en) | Equipment abnormal condition voiceprint analysis algorithm based on compressed neural network | |
| CN109829887B (en) | Image quality evaluation method based on deep neural network | |
| CN118260637A (en) | System and device for emotion semantic analysis management based on artificial intelligence | |
| CN117942038A (en) | Sleep stage identification system integrating cross prediction and discrimination tasks | |
| CN117707089A (en) | Production workshop abnormity monitoring system based on Internet of things | |
| CN116304112A (en) | Intelligent monitoring method based on big data technology | |
| CN112419304B (en) | Multi-stage target detection method and device for one-dimensional data | |
| CN115391306A (en) | Data center abnormal state detection method based on LSTM self-encoder | |
| CN114821174A (en) | Power transmission line aerial image data cleaning method based on content perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |