CN116013362A

CN116013362A - Method and device for determining fault type, computer equipment and readable storage medium

Info

Publication number: CN116013362A
Application number: CN202211582372.0A
Authority: CN
Inventors: 董勤波
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2022-12-09
Filing date: 2022-12-09
Publication date: 2023-04-25

Abstract

The embodiment of the application provides a fault type determining method, a fault type determining device, computer equipment and a readable storage medium, and relates to the technical field of fault voiceprint monitoring. The method comprises the following steps: determining the similarity between sample voiceprint vectors of different devices of the same type; dividing different devices of the same type into a plurality of subtypes according to the similarity; acquiring device voiceprint vectors of target devices in different devices of the same type; if the equipment voiceprint vector of the target equipment is determined to be the fault voiceprint vector, determining the fault type of the target equipment according to the corresponding relation in the target fault sample library. According to the method provided by the embodiment of the invention, the equipment of the same type is classified through the similarity among the sample voiceprint vectors of the different equipment of the same type, and the classified equipment shares the fault sample library, so that the influence of the difference among the different equipment on the fault type determining process can be reduced, and the accuracy of determining the equipment fault type is improved.

Description

Method and device for determining fault type, computer equipment and readable storage medium

Technical Field

The present invention relates to the field of fault voiceprint monitoring technologies, and in particular, to a method and apparatus for determining a fault type, a computer device, and a readable storage medium.

Background

During operation of the device, since vibrations occur between the body and the firmware, between the parts or between the parts themselves, so as to generate audio data which can characterize the state of the device itself, and this audio data is unique to the device, and this audio data can also be measured and analyzed by an electroacoustic instrument, we call the characteristic of the audio data carried to characterize the operation state of the device as voiceprint. The voiceprint contains a large amount of equipment state information, so that the voiceprint changes along with the change of the running state of the equipment, and the voiceprint can be used as an important characteristic parameter for abnormality detection and fault analysis of the equipment.

In the prior art, the voiceprint of the equipment is utilized for abnormality detection and fault analysis, and a fault sample library is required to be established in advance. Based on the principle of fault sharing, the equipment of the same type shares a fault sample library, and the collected equipment voiceprints are compared with fault sample information in the same fault sample library to determine the fault type of the equipment. However, even for the same type of equipment, due to the difference in component size, operating power, etc., there is a large difference in voiceprint when it operates, so that the accuracy of determining the type of failure is not high.

Therefore, how to improve the accuracy of determining the fault type is a technical problem to be solved.

Disclosure of Invention

The application provides a fault type determining method, a fault type determining device, computer equipment and a readable storage medium, which are used for improving the accuracy of fault type determination.

An embodiment of the present application provides a method for determining a fault type, including: and determining the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices normally operate. Different devices of the same type are divided into a plurality of sub-types according to the similarity. Wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types. And acquiring the device voiceprint vectors of the target devices in different devices of the same type. If the equipment voiceprint vector of the target equipment is determined to be the fault voiceprint vector, determining the fault type of the target equipment according to the corresponding relation in the target fault sample library; the target fault sample library is a fault sample library corresponding to the subtype to which the target device belongs.

According to the method for determining the fault type, according to the similarity between the sample voiceprint vectors of the same type of equipment, equipment with the similarity larger than or equal to the first preset threshold value in the same type of equipment is divided into one subtype. It can be seen that the difference of the devices in each subtype after classification is smaller, that is, the sound characteristics of the devices in each subtype are also the same, and then the devices in each subtype correspond to a fault sample library. Therefore, when the fault type of the target equipment is judged, the fault type of the target equipment is determined according to the corresponding relation in the target fault sample library, so that the fault type of the target equipment can be more accurate. The method provided by the embodiment of the invention can reduce the influence of the difference between different devices on the fault type determining process, and further can well improve the accuracy of determining the fault type.

With reference to the first implementation manner of the first aspect, the step of acquiring the device voiceprint vectors of the target device in different devices of the same type includes: acquiring device audio data of a target device; determining a device voiceprint vector corresponding to the device audio data; the step of determining that the device voiceprint vector of the target device is a failed voiceprint vector includes: obtaining a first similarity set according to the similarity of the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; and if the maximum similarity in the first similarity set is smaller than a first preset threshold value, determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector.

With reference to the second implementation manner of the first aspect, the step of determining a device voiceprint vector corresponding to device audio data includes: extracting characteristic data of the equipment audio data; acquiring equipment state parameters of target equipment, wherein the equipment state parameters comprise K groups of basic frequencies and target formant frequencies of the target equipment, the target formant frequencies are resonance frequencies which are arranged at the front K bits when the resonance frequencies generated by the target equipment are arranged in the order from big to small, and K is a positive integer; and obtaining the equipment voiceprint vector corresponding to the equipment audio data according to the characteristic data and the equipment state parameters.

With reference to the third implementation manner of the first aspect, the step of obtaining, according to the feature data and the device state parameter, a device voiceprint vector corresponding to the device audio data includes: and after the characteristic data and the equipment state parameters are spliced, inputting the characteristic data and the equipment state parameters into a neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

With reference to the fourth implementation manner of the first aspect, the step of obtaining, according to the feature data and the device state parameter, a device voiceprint vector corresponding to the device audio data includes: the characteristic data and the equipment state parameters are spliced and then input into a first part of a neural network to obtain network hidden layer characteristics, wherein the neural network comprises the first part and a second part; and (3) splicing the equipment state parameters and the network hidden layer characteristics, and inputting the spliced equipment state parameters and the network hidden layer characteristics into a second part of the neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

With reference to the fifth implementation manner of the first aspect, the step of determining, according to a correspondence in the target fault sample library, a fault type of the target device includes: obtaining a second similarity set according to the similarity of the fault voiceprint vector and each fault voiceprint vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold value, determining a target fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity in the target fault sample library according to the corresponding relation in the target fault sample library, wherein the target fault type is the fault type of the target equipment.

With reference to the sixth implementation manner of the first aspect, the step of determining, according to a correspondence in the target fault sample library, a fault type of the target device further includes: if the maximum similarity in the second similarity set is smaller than a second preset threshold, outputting prompt information, wherein the prompt information is used for prompting the input of the fault type; in response to acquiring the input fault type, determining that the fault type of the target equipment is the input fault type; the method further comprises the steps of: and writing the corresponding relation between the fault voiceprint vector and the input fault type into a target fault sample library.

A second aspect of the embodiments of the present application provides a fault type determining apparatus, including: the first determining module is used for determining the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices normally operate; the classification module is used for dividing different devices of the same type into a plurality of subtypes according to the similarity; wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types; the acquisition module is used for acquiring the equipment voiceprint vectors of the target equipment in different equipment of the same type; the second determining module is used for determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector, and determining the fault type of the target equipment according to the corresponding relation in the target fault sample library; the target fault sample library is a fault sample library corresponding to the subtype to which the target device belongs.

With reference to the first implementation manner of the second aspect, the acquiring module is further configured to: acquiring device audio data of a target device; determining a device voiceprint vector corresponding to the device audio data; the second determination module is further configured to: obtaining a first similarity set according to the similarity of the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; and if the maximum similarity in the first similarity set is smaller than a first preset threshold value, determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector.

With reference to the second implementation manner of the second aspect, the acquiring module is further configured to: extracting characteristic data of the equipment audio data; acquiring equipment state parameters of target equipment, wherein the equipment state parameters comprise K groups of basic frequencies and target formant frequencies of the target equipment, the target formant frequencies are resonance frequencies which are arranged at the front K bits when the resonance frequencies generated by the target equipment are arranged in the order from big to small, and K is a positive integer; and obtaining the equipment voiceprint vector corresponding to the equipment audio data according to the characteristic data and the equipment state parameters.

With reference to the third implementation manner of the second aspect, the acquiring module is further configured to: and after the characteristic data and the equipment state parameters are spliced, inputting the characteristic data and the equipment state parameters into a neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

With reference to the fourth implementation manner of the second aspect, the acquiring module is further configured to: the characteristic data and the equipment state parameters are spliced and then input into a first part of a neural network to obtain network hidden layer characteristics, wherein the neural network comprises the first part and a second part; and (3) splicing the equipment state parameters and the network hidden layer characteristics, and inputting the spliced equipment state parameters and the network hidden layer characteristics into a second part of the neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

With reference to the fifth implementation manner of the second aspect, the second determining module is further configured to: obtaining a second similarity set according to the similarity of the fault voiceprint vector and each fault voiceprint vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold value, determining a target fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity in the target fault sample library according to the corresponding relation in the target fault sample library, wherein the target fault type is the fault type of the target equipment.

With reference to the sixth implementation manner of the second aspect, the second determining module is further configured to: if the maximum similarity in the second similarity set is smaller than a second preset threshold, outputting prompt information, wherein the prompt information is used for prompting the input of the fault type; in response to acquiring the input fault type, determining that the fault type of the target equipment is the input fault type; the fault type determining device further comprises a writing module, which is used for writing the corresponding relation between the fault voiceprint vector and the input fault type into the target fault sample library.

A third aspect of the embodiments of the present application provides a computer device, including: a processor and a memory; the memory is used for storing the program codes by the computer and transmitting the computer program codes to the processor; the processor is configured to perform the above-described method of determining a fault type according to instructions in the computer program code.

A fourth aspect of the embodiments of the present application provides a computer readable storage medium, on which a computer program is stored, which when run on a computer device causes the device to perform the above-described method of determining a fault type.

The advantages described in the second to fourth aspects may be referred to for analysis of the advantages of the first aspect, and are not described here.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate and do not limit the invention.

Fig. 1 is a schematic structural diagram of a fault type determining system according to an embodiment of the present application;

fig. 2 is a method flowchart of a method for determining a fault type according to an embodiment of the present application;

fig. 3 is a flowchart of a method for obtaining a voiceprint vector of a device according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a device voiceprint vector generating process according to an embodiment of the present application;

fig. 5 is a flow chart of a method for determining a fault type according to an embodiment of the present application;

Fig. 6 is a schematic structural diagram of a fault type determining apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

The terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.

In the description of the present application, it should be noted that, unless explicitly stated and limited otherwise, the terms "connected," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context. In addition, when describing a pipeline, the terms "connected" and "connected" as used herein have the meaning of conducting. The specific meaning is to be understood in conjunction with the context.

In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

During the running process of the equipment, the machine body and the firmware, the parts or the parts themselves vibrate, so that audio data which can represent the state of the equipment are generated, and the audio data are unique to the equipment and can be measured and analyzed through an electroacoustical instrument, so that the characteristic carried by the audio data and representing the running state of the equipment is called voiceprint. The voiceprint contains a large amount of equipment state information, so that the voiceprint changes along with the change of the running state of the equipment, and the voiceprint can be used as an important characteristic parameter for abnormality detection and fault analysis of the equipment.

In the related art, the voice print of the equipment is utilized for abnormality detection and fault analysis, and a fault sample library is required to be established in advance. Based on the principle of fault sharing, the equipment of the same type shares a fault sample library, and the collected equipment voiceprints are compared with fault sample information in the same fault sample library to determine the fault type of the equipment. However, even for the same type of equipment, due to the difference in component size, operating power, etc., there is a large difference in voiceprint when it operates, so that the accuracy of determining the type of failure is not high.

Wherein the same type of device may be devices having the same function or having the same physical form. For example, in one possible embodiment, the same type of device is a pump, which is a machine for delivering or pressurizing liquid, and may be classified into a volumetric pump, a vane pump, etc. according to different working principles, but all have the same function, and may be referred to as the same type of device.

The voiceprint is a sound wave spectrum which is displayed by an electroacoustical instrument and carries speech information, and abnormality detection and fault analysis are performed according to the voiceprint of the device, namely, the voiceprint feature (feature) of the voiceprint of the device is essentially utilized for analysis, namely, mathematical operation is needed to be performed on audio data of the device, and a group of feature description vectors { x1, x2, }, xn (n is a positive integer) are extracted from the audio data. Therefore, for convenience of description, in the embodiment of the present application, the voiceprint may also be referred to as a voiceprint vector.

Aiming at the problems, the method for determining the fault type is introduced, and can divide the equipment of the same type into a plurality of subtypes according to the similarity between the sample voiceprint vectors of the equipment of the same type, and the equipment in each subtype shares a fault sample library, so that the accuracy of determining the fault type can be improved well.

The method for determining the fault type provided by the embodiment of the application can be applied to a fault type determining system shown in fig. 1. As shown in fig. 1, the fault type determining system includes: processing means 101, audio acquisition means 102 and device 103. The processing device 101 may establish a connection with the audio acquisition device 102 and the device 103 through a wired network or a wireless network, and the audio acquisition device 102 and the device 103 may establish a connection through a wired network or a wireless network.

The device 103 refers to a device with a specific function or a specific physical form, and generates device audio data by vibrating between a machine body and firmware, parts or between parts themselves during operation. Such as electrical equipment for delivering electrical energy, pumps for delivering liquids, etc. The embodiment of the present application does not limit the specific device configuration of the device 103.

The audio collection device 102 is configured to collect device audio data generated during the operation of the device 103, and send the device audio data to the processing device 101. The processing apparatus 101 is caused to generate a corresponding device voiceprint vector from the device audio data, and then determine whether the device voiceprint vector is a faulty voiceprint vector according to the similarity of the device voiceprint vector to a pre-stored sample voiceprint vector. If the fault voiceprint vector is determined to be the fault voiceprint vector, determining the fault type corresponding to the equipment 103 according to a fault sample library preset in the processing device 101.

The audio collection device 102 may be deployed separately from the apparatus 103, or the audio collection device 102 may be disposed on the apparatus 103 so as to collect audio data generated by the apparatus 103 when it is running. The audio collection device 102 may be a microphone collection system or an acoustic array device, which is not limited in this embodiment of the present application.

In some embodiments, the processing device 101 may be a server, and the specific implementation of the server is not limited in the embodiments of the present application. The server may be a single server, or may be a server cluster formed by a plurality of servers. In some implementations, the server cluster may also be a distributed cluster.

In other embodiments, the processing apparatus 101 may also be a computer device, and the specific form of the computer device is not limited in the embodiments of the present application. For example, the computer device may be a terminal apparatus. Wherein the terminal device may be referred to as: a terminal, a User Equipment (UE), a terminal device, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, a user equipment, or the like. The terminal device may be a mobile phone, an augmented reality (augmented rea l i ty, AR) device, a virtual reality (v i rtua l rea l i ty, VR) device, a tablet computer, a notebook computer, an ultra mobile personal computer (u l tra-mob i l e persona l computer, UMPC), a netbook, a personal digital assistant (persona l d i g i ta l ass i stant, PDA), or the like.

It should be noted that fig. 1 is merely an exemplary application scenario showing that the fault type determining system includes only one device, and in a practical application process, the fault type determining system may include a plurality of devices 103 and corresponding audio acquisition devices 102, which is not limited in any way in the embodiments of the present application.

It should be noted that the configuration shown in fig. 1 does not constitute a limitation of the fault type determination system, and in other embodiments, the system may include fewer or more components than shown, or certain components may be combined, or different arrangements of components, which are not limited in any way by the embodiments of the present application.

Fig. 2 is a flow chart illustrating a fault type determining method according to an embodiment of the present application, which may be applied to the processing apparatus 101 in the system shown in fig. 1. As shown in fig. 2, the method comprises the steps of:

s201, determining the similarity between sample voiceprint vectors of different devices of the same type.

The sample voiceprint vector is a voiceprint vector corresponding to audio data generated during normal operation of the device. The running state of the equipment can be divided into normal running and fault states, and the fault types of the equipment in the fault states are various, and voiceprint vectors corresponding to audio data generated in the fault states are also various, so that the equipment cannot be used as the basis for equipment classification. Therefore, the embodiment of the application adopts the similarity between the sample voiceprint vectors of the device as the basis of classification.

For example, if the different devices of the same type are pump a, pump B, and pump C, respectively. Then the similarity of the sample voiceprint vectors of pump a and pump B, the similarity of the sample voiceprint vectors of pump B and pump C, and the similarity of the sample voiceprint vectors of pump a and pump C need to be determined.

In the embodiment of the present application, the algorithm for calculating the similarity may be, but is not limited to: cosine similarity (cos i ne s im i l ar i ty) scoring, probabilistic linear discriminant analysis (probab i l i st i c l i near d i scr im i nant ana l ys i s, PLDA) scoring, or cosine distance (cos i ne d i stance), etc.

For example, using cosine similarity (cos i ne s im i l ar i ty) algorithm to calculate similarity, since the voiceprint vector is an n-dimensional vector, it is assumed that voiceprint vector A is { A } ₁ ,A,…,A _n Voiceprint vector B is { B } ₁ ,B ₂ ,…,B _n The similarity of vector a and vector B is calculated as follows:

the obtained number is the similarity score of the vector A and the vector B.

S202, different devices of the same type are divided into a plurality of subtypes according to the similarity.

The similarity between different devices in one subtype is larger than or equal to a first preset threshold value, each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types.

Specifically, after the similarity between the sample voiceprint vectors corresponding to the devices is obtained, the devices are classified according to the similarity, so that the similarity between the sample voiceprint vectors is greater than or equal to a first preset threshold value, and the devices serving as the devices of the same subtype, namely, the similarity between the devices in each subtype is greater than or equal to the first preset threshold value. Each sub-type of equipment can share a fault sample library after classification, so that the accuracy of determining the fault type can be well improved in the process of determining the fault type.

The first preset threshold is preset by the system, and can be set according to requirements in the actual application process, which is not limited in the embodiment of the application. For example, as a feasibility implementation, the first preset threshold is 75%, and the similarity between devices in each subtype after classification is greater than or equal to 75%.

As a possible implementation manner, S202 may specifically be: and generating a similarity matrix by using the similarity between the sample voiceprint vectors of different devices, sending the similarity matrix and a first preset threshold value into a clustering algorithm, clustering the similarity in the similarity matrix by using the first preset threshold value by the clustering algorithm, and then clustering the devices according to the devices corresponding to the similarity, so that the different devices of the same type are divided into a plurality of sub-types.

The clustering algorithm used may be hierarchical clustering (agg l omerat i ve h i erarch i ca lc l uster i ng, AHC) algorithm or spectral clustering algorithm, etc., and may be set according to requirements in the actual application process, which is not limited in the embodiment of the present application.

S203, acquiring device voiceprint vectors of target devices in different devices of the same type.

Because the audio data is sent out in the operation process of the target device, anomaly detection and fault analysis are performed according to the audio data of the target device, that is, analysis is essentially performed by using voiceprint features (features) of the audio data of the target device, that is, mathematical operations need to be performed on the audio data of the target device, a set of feature description vectors { x1, x2, & gt, xn } are extracted from the audio data, that is, device voiceprint vectors, and then the current state of the target device is determined according to the device voiceprint vectors.

As a feasibility implementation, S203 may include: (11) and (12).

(11) Device audio data of the target device is acquired.

(12) And determining the equipment voiceprint vector corresponding to the equipment audio data.

And in the running process of the target equipment, acquiring sound generated by vibration between the machine body of the target equipment and the firmware, between parts or between the parts, taking the sound as equipment audio data, and then determining the corresponding equipment voiceprint vector according to the equipment audio data. It should be appreciated that determining the device voiceprint vector to which the device audio data corresponds may be performed in a manner conventional in the art, and applicant does not make any undue limitations herein.

In some embodiments, since the state change is complex during the operation of the device, the target device may be in different states along with the change of time, and thus, in the process of forming the device voiceprint vector, the device state parameter may be introduced as a supplement to the device audio data, so that the generated device voiceprint vector is related to the operation state of the device.

In the embodiment of the present application, the device state parameters are some parameters for characterizing the device state. For example, the motion parameters (displacement, velocity, acceleration) and magnitude parameters (peak, average and effective values) generated by vibrations of the device during operation.

As a feasibility implementation, as shown in fig. 3, the step of determining a device voiceprint vector corresponding to device audio data may include:

s301, extracting characteristic data of the device audio data.

The method for extracting the feature data of the audio data of the device may be a method commonly used in the art, for example, extracting based on the features (fi l ter bank, fbank) of the filter bank, or extracting the features (me l-frequency cepstra l coeff i c i ents, MFCC) of the mel-frequency coefficient, and in the practical application process, the method may be selected according to the requirement, which is not limited in any way in the embodiments of the present application.

In some embodiments, the Fbank feature has a higher correlation with the device audio data of the target device, as the Fbank feature retains more of the original audio data. Therefore, as a feasibility implementation manner, the method for extracting the feature data of the device audio data is Fbank feature extraction, and 128-dimensional Fbank feature data is obtained.

S302, acquiring device state parameters of the target device.

In order to enable the equipment state parameter to be capable of representing the equipment state more accurately, the embodiment of the application also discloses an implementation mode for obtaining the equipment state parameter of the target equipment, which specifically comprises the following steps:

k groups of fundamental frequencies and target formant frequencies of the device are extracted to characterize the operating state of the device. The target formant frequency is the resonance frequency arranged in the first K bits when the resonance frequencies generated by the devices are arranged in the order of the frequencies from the large to the small. K is a positive integer, and as a feasible implementation manner, K is 4, and 4 groups of fundamental frequencies plus the target formant frequencies are used for representing the running state of the equipment, which is not limited in any way in the embodiment of the application.

According to the implementation mode, the basic frequency and the target formant frequency which are arranged in the front K bits are used as the equipment state parameters, and the equipment state parameters can accurately represent the equipment state.

S303, obtaining the equipment voiceprint vector corresponding to the equipment audio data according to the characteristic data and the equipment state parameters.

The device state parameters are introduced to supplement the device audio data, so that the generated device voiceprint vector can better represent the running state of the device. When the running state of the equipment is judged according to the voiceprint vector of the equipment, the equipment can be more accurate.

The neural network is a pre-trained network model, including but not limited to a residual network (RestNet) and a time delay neural network (t ime de l ay neura l network, TDNN), and in the practical application process, a specific network model can be used according to the needs, which is not limited in the embodiments of the present application.

It should be noted that the embodiment of the present application does not specifically limit the sequence of S301 and S302. As a feasibility implementation, S301 may be performed first and S302 may be performed later; s302 may be performed first and S301 may be performed later as a feasibility implementation.

As a feasibility implementation, S303 may be specifically implemented as: and after the characteristic data and the equipment state parameters are spliced, inputting the characteristic data and the equipment state parameters into a neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

The characteristic data and the equipment state parameters are spliced and then input into the neural network, so that the generated equipment voiceprint vector is related to the equipment running state, and the running state of the equipment can be judged more accurately.

In some embodiments, the dimension of the feature data of the device audio data is 128 dimensions, the dimension of the device state parameter is only one digit, and the information contained in the device state parameter is used for representing the running state of the device and is not easy to learn by the neural network. That is, the neural network is not easy to model the device state parameters in the process of generating the device voiceprint vector.

As another possible implementation, as shown in connection with fig. 4, S303 may include the steps of:

s3031, the characteristic data and the equipment state parameters are spliced and then input into a first part of the neural network, so that the network hidden layer characteristics are obtained.

S3032, the equipment state parameters and the network hidden layer characteristics are spliced and then input into a second part of the neural network, so that equipment voiceprint vectors corresponding to the equipment audio data are obtained.

Wherein the neural network includes a first portion and a second portion. In connection with fig. 4, the dimension of the feature data is 128 dimensions, the dimension of the device state parameter is only one digit, and the information contained in the device state parameter characterizes the operation state of the device, so that the neural network is not easy to model the device state parameter. And therefore, the characteristic data and the equipment state parameters are spliced, and the characteristic data and the equipment state parameters are sent to a first part of the neural network for calculation after the splicing is successful, so that the network hidden layer characteristics are obtained. And splicing the equipment state parameters and the network hidden layer characteristics again to improve the modeling capacity of the neural network, and sending the equipment state parameters and the network hidden layer characteristics into a second part of the neural network to calculate after the equipment state parameters and the network hidden layer characteristics are successfully spliced to obtain the equipment voiceprint vector. The generated equipment voiceprint vector is related to the running state of the equipment, and the running state of the equipment can be more accurate when the running state of the equipment is judged according to the equipment voiceprint vector.

S204, if the equipment voiceprint vector of the target equipment is determined to be the fault voiceprint vector, determining the fault type of the target equipment according to the corresponding relation in the target fault sample library.

The target fault sample library is a fault sample library corresponding to a subtype to which the target equipment belongs. Specifically, since the different devices of the same type are already classified in S202, each device has its corresponding target fault sample library. If the equipment voiceprint vector of the target equipment is determined to be the fault voiceprint vector, the fault state of the target equipment is indicated, so that the fault type of the target equipment is determined by using the corresponding relation between the fault voiceprint vector and the fault type in the target fault sample library.

As a feasibility implementation, the step of determining that the device voiceprint vector of the target device is a fault voiceprint vector includes: (21) and (22).

(21) Obtaining a first similarity set according to the similarity of the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs;

(22) And if the maximum similarity in the first similarity set is smaller than a first preset threshold value, determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector.

Specifically, similarity calculation is performed on the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs, so as to obtain a first similarity set. If one of the similarities in the first similarity set is greater than a preset threshold, the device voiceprint vector is similar to the sample voiceprint vector of a certain device, so that the device is judged to be in a normal running state. If all the similarities in the first similarity set are smaller than a preset threshold, the fact that the equipment voiceprint vector is dissimilar to sample voiceprint vectors of all the equipment in the subtype to which the target equipment belongs is indicated, the equipment voiceprint vector is judged to be a fault voiceprint vector, and the target equipment is in a fault state.

Before determining the fault type of the target device, determining whether the target device is in a fault state, and if the target device is in the fault state, further determining the fault type of the target device according to the fault voiceprint vector. Therefore, whether the equipment is in a fault state is judged, the fault type is determined after the equipment is in the fault state, a large number of normal voiceprint vectors can be screened out, the fault type does not need to be frequently determined, and the accuracy of determining the fault type can be improved.

As a feasibility implementation manner, the step of determining the fault type of the target device according to the correspondence in the target fault sample library includes: (31) and (32).

(31) And obtaining a second similarity set according to the similarity of the fault voiceprint vector and each fault voiceprint vector in the target fault sample library.

(32) If the maximum similarity in the second similarity set is greater than or equal to a second preset threshold value, determining a target fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity in the target fault sample library according to the corresponding relation in the target fault sample library.

The second preset threshold is preset by the system, and can be set according to requirements in the actual application process, which is not limited in the embodiment of the application. For example, as a feasibility implementation manner, the second preset threshold is 80%, and if the maximum similarity in the second similarity set is greater than or equal to 80%, the target fault type is the fault type corresponding to the fault voiceprint vector that generates the maximum similarity.

The target fault type is the fault type of the target equipment.

Specifically, similarity calculation is carried out on the fault voiceprint vectors and all fault voiceprint vectors in the target fault sample library, so that a second similarity set is obtained. If the maximum similarity in the second similarity set is greater than a second preset threshold, the fault voiceprint vector is the most similar to the fault voiceprint vector generating the maximum similarity, so that the fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity is used as the target fault type. The obtained target fault type is the most conforming fault type in the target fault sample library, namely the obtained fault type of the target equipment is the most accurate.

In some embodiments, the maximum similarity in the second similarity set may be smaller than the second threshold, that is, all the similarities in the second similarity set are smaller than the second threshold, which indicates that the fault voiceprint vector of the target device is not similar to the fault voiceprint vectors in the fault sample library, and the fault type of the target device cannot be determined.

As a feasibility implementation manner, the step of determining the fault type of the target device according to the correspondence in the target fault sample library further includes: (41) and (42).

(41) And if the maximum similarity in the second similarity set is smaller than a second preset threshold, outputting prompt information.

(42) And in response to acquiring the input fault type, determining that the fault type of the target device is the input fault type.

The prompt information is used for prompting the input of the fault type. Specifically, the prompt information can prompt that the fault voiceprint vector processed at present is not recorded in the fault sample library, so that a background personnel analyzes the fault type corresponding to the fault voiceprint vector and inputs the fault type.

The embodiment of the application does not specifically limit the output form of the prompt information, and as a feasibility implementation mode, the prompt information can be output in a video mode; as another feasibility implementation, the hint information may be output in audio.

If the maximum similarity in the second similarity set is smaller than the second threshold, that is, all the similarities in the second similarity set are smaller than the second threshold, it is indicated that the fault voiceprint vector of the target device is not similar to the fault voiceprint vector in the fault sample library, the target device is in a fault state, but no corresponding fault type exists, and a user or a field expert is required to confirm the fault type of the target device through manual confirmation.

As a feasibility implementation, the method further comprises:

and writing the corresponding relation between the fault voiceprint vector and the input fault type into a target fault sample library.

And writing the corresponding relation between the fault voiceprint vector and the input fault type into a target fault sample library, so that when similar fault voiceprint vectors are generated by equipment in the subtype to which the subsequent target equipment belongs, the fault type can be identified as the input fault type, and the repeated confirmation by manpower is not needed.

It should be understood that, in the fault sample library provided in the embodiments of the present application, in the process of identifying the fault type by using the fault sample library, the correspondence between the fault voiceprint vector and the fault type may not be stored in the target fault sample library. Then the similarity between the fault voiceprint vector and each fault voiceprint vector in the target fault sample library is calculated, the obtained second similarity set is empty, that is, the maximum similarity in the second similarity set is smaller than a second preset threshold value, and the method can also be used for outputting prompt information to prompt a background personnel to input the fault type.

That is, the method provided by the embodiment of the application supports the fault sample library to perform data cold start. The operation can be started under the condition that the fault sample library has no fault sample, and the fault sample library is continuously expanded in the operation process to expand the fault type.

In some embodiments, a sample database may be provided for storing sample voiceprint vectors of different devices of the same type, and in performing S201, the devices may be classified directly according to the sample voiceprint vectors in the sample database. So that the classified devices of the same sub-type correspond to a fault sample database and the classified devices of the same sub-type also correspond to a sub-sample database. Therefore, when judging whether the equipment voiceprint vector of the target equipment is the fault voiceprint vector, judging whether the equipment voiceprint vector is the fault voiceprint vector according to the sample voiceprint vector in the sub-sample database corresponding to the target equipment. Because the similarity between the classified devices is higher, the devices of the same subtype correspond to a sub-sample database, and whether the voiceprint vector of the device is a fault voiceprint vector can be judged more accurately.

Referring to fig. 5, fig. 5 is a flowchart illustrating a method for determining a fault type according to an exemplary embodiment of the present application.

When the equipment operates, sample data under normal operation of the equipment is firstly sent into a voiceprint extraction model to obtain sample voiceprint vectors corresponding to the normal sample data, and the sample voiceprint vectors are stored in a sample database as sample data. Voiceprint recognition is then performed using the following steps:

S501, obtaining the device voiceprint vector according to the device audio data.

And sending the equipment audio data generated by the equipment operation and the equipment state parameters of the current state of the equipment into a voiceprint extraction model, and outputting the model to obtain the equipment voiceprint vector.

It should be understood that the voiceprint extraction model may be the neural network described above, or may be another pre-trained model, which is not limited in this embodiment of the present application.

S502, performing similarity comparison on the equipment voiceprint vector and a sample voiceprint vector in a sample database.

Specifically, similarity calculation is performed on the voiceprint vector of the device and the sample voiceprint vector to obtain a first similarity set, and then all data in the first similarity set are compared with a first system threshold.

S503, judging whether the equipment voiceprint vector is a fault voiceprint vector.

If one of the similarities in the first similarity set is greater than a preset threshold, the voiceprint vector is similar to sample data in the sample database, so that the equipment is judged to be in a normal running state. If all the similarities in the first similarity set of the voiceprint vector and the sample voiceprint vector in the sample database are smaller than a preset threshold, the voiceprint vector is indicated to be a fault voiceprint, and the equipment is in a fault state.

S5041, if the equipment is in a normal operation state, the output equipment operates normally.

S5042, if the equipment voiceprint vector is a fault voiceprint vector, comparing the fault voiceprint vector with fault voiceprint vectors in a target fault sample library.

Specifically, similarity calculation can be performed on fault voiceprint vectors in the fault voiceprint and target fault sample library to obtain a second similarity set, and then all data in the second similarity set are compared with a system threshold value.

S505, judging whether the fault is known.

If the target fault sample library has no fault sample data, judging that the fault voiceprint is an unknown fault. If the maximum similarity element in the second similarity set is greater than or equal to a preset threshold value, judging that the fault voiceprint is similar to the fault voiceprint vector generating the maximum similarity, wherein the fault voiceprint is a known fault. If all similarity elements in the second similarity set are smaller than a preset threshold value, judging that the fault voiceprint is dissimilar to all fault voiceprint vectors in the target fault sample library, wherein the fault voiceprint is not a known fault and is an unknown fault.

S5061, if the fault voiceprint is a known fault, outputting a fault type corresponding to the fault voiceprint vector.

And calling the fault type corresponding to the fault voiceprint vector in the target fault sample library, and outputting the fault type as the fault type of the fault voiceprint.

S5062, if the failed voiceprint is an unknown failure, outputting the failed voiceprint as an unknown failure.

If the fault voiceprint is an unknown fault, the system outputs prompt information to prompt that the fault is the unknown fault, waits for human confirmation and inputs the fault type, and names the fault.

S507, after receiving the fault type, storing the fault voiceprint vector and the fault type into a target fault sample library.

And naming the unknown fault by a user or a field expert through manual confirmation, and taking the name as the fault type corresponding to the fault voiceprint vector. And the fault voiceprint vector and the fault type are stored in a target fault sample library so as to expand the target fault sample library, so that the similar fault voiceprint vector is monitored again later, the system can judge the fault type in time, and the fault type does not need to be confirmed again manually.

The embodiment of the application also provides a device for determining a fault type, as shown in fig. 6, the determining device 60 includes: a first determination module 61, a classification module 62, an acquisition module 63 and a second determination module 64.

The first determining module 61 is configured to determine a similarity between sample voiceprint vectors of different devices of the same type, where the sample voiceprint vectors are voiceprint vectors when the devices are operating normally;

a classification module 62, configured to divide different devices of the same type into a plurality of sub-types according to the similarity; wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types;

an obtaining module 63, configured to obtain device voiceprint vectors of target devices in different devices of the same type;

a second determining module 64, configured to determine that the device voiceprint vector of the target device is a fault voiceprint vector, and determine a fault type of the target device according to a correspondence in the target fault sample library; the target fault sample library is a fault sample library corresponding to the subtype to which the target device belongs.

As one feasibility implementation, the acquisition module is further configured to: acquiring device audio data of a target device; determining a device voiceprint vector corresponding to the device audio data; the second determination module is further configured to: obtaining a first similarity set according to the similarity of the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; and if the maximum similarity in the first similarity set is smaller than a first preset threshold value, determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector.

As one feasibility implementation, the acquisition module is further configured to: extracting characteristic data of the equipment audio data; acquiring equipment state parameters of target equipment, wherein the equipment state parameters comprise K groups of basic frequencies and target formant frequencies of the target equipment, the target formant frequencies are resonance frequencies which are arranged at the front K bits when the resonance frequencies generated by the target equipment are arranged in the order from big to small, and K is a positive integer; and obtaining the equipment voiceprint vector corresponding to the equipment audio data according to the characteristic data and the equipment state parameters.

As a feasible implementation, the acquisition module is further configured to: and after the characteristic data and the equipment state parameters are spliced, inputting the characteristic data and the equipment state parameters into a neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

As a feasible implementation, the acquisition module is further configured to: the characteristic data and the equipment state parameters are spliced and then input into a first part of a neural network to obtain network hidden layer characteristics, wherein the neural network comprises the first part and a second part; and (3) splicing the equipment state parameters and the network hidden layer characteristics, and inputting the spliced equipment state parameters and the network hidden layer characteristics into a second part of the neural network to obtain equipment voiceprint vectors corresponding to the equipment audio data.

As one feasibility implementation, the second determining module is further configured to: obtaining a second similarity set according to the similarity of the fault voiceprint vector and each fault voiceprint vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold value, determining a target fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity in the target fault sample library according to the corresponding relation in the target fault sample library, wherein the target fault type is the fault type of the target equipment.

As one feasibility implementation, the second determining module is further configured to: if the maximum similarity in the second similarity set is smaller than a second preset threshold, outputting prompt information, wherein the prompt information is used for prompting the input of the fault type; in response to acquiring the input fault type, determining that the fault type of the target equipment is the input fault type; the fault type determining device further comprises a writing module, which is used for writing the corresponding relation between the fault voiceprint vector and the input fault type into the target fault sample library.

The embodiment of the present application further provides a computer device, please refer to fig. 7, the device includes a memory 71 and a processor 72, wherein the memory 71 is used for storing computer program codes and transmitting the computer program codes to the processor 72; the processor 72 is operative to determine the type of fault from the audio data generated by the apparatus in accordance with instructions in the computer program code.

Alternatively, the memory 71 may be a non-transitory computer-readable storage medium, which may be, for example, read-on-l y memory (ROM), random-access memory (random access memory, RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like, to which the embodiments of the present application do not impose any limitation.

The processor 72 may be a central processing unit (centra l process i ng un i t, CPU), a general purpose processor network processor (network processor, NP), a digital signal processor (d i g i ta l s i gna l process i ng, DSP), a microprocessor, a microcontroller, a programmable logic device (programmab l e l og i c dev i ce, PLD), or any combination thereof, to which the embodiments of the present application are not limited in any way.

Embodiments of the present application also provide a computer program product comprising one or more instructions stored in the memory 71 of a computer device for execution by the processor 72 to perform the various processes of the embodiments described above.

The present application also provides a computer-readable storage medium including computer-executable instructions that, when executed on a computer, cause the computer to perform a method as provided in the above embodiments.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts shown as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a device (may be a single-chip microcomputer or the like) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of determining a type of fault, the method comprising:

determining the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices normally operate;

dividing different devices of the same type into a plurality of subtypes according to the similarity; wherein the similarity between different devices in one of the sub-types is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types;

acquiring the equipment voiceprint vectors of target equipment in different equipment of the same type;

if the equipment voiceprint vector of the target equipment is determined to be the fault voiceprint vector, determining the fault type of the target equipment according to the corresponding relation in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target equipment belongs.

2. The method of claim 1, wherein the step of obtaining the device voiceprint vector for the target device of the different devices of the same type comprises:

acquiring equipment audio data of the target equipment;

determining the equipment voiceprint vector corresponding to the equipment audio data;

the step of determining that the device voiceprint vector of the target device is a failed voiceprint vector includes:

obtaining a first similarity set according to the similarity of the equipment voiceprint vector and the sample voiceprint vector of each equipment in the subtype to which the target equipment belongs;

and if the maximum similarity in the first similarity set is smaller than the first preset threshold value, determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector.

3. The method of claim 2, wherein the step of determining the device voiceprint vector for the device audio data comprises:

extracting characteristic data of the equipment audio data;

acquiring equipment state parameters of the target equipment, wherein the equipment state parameters comprise K groups of basic frequencies and target formant frequencies of the target equipment, the target formant frequencies are the resonance frequencies which are arranged in front K bits when the resonance frequencies generated by the target equipment are arranged in the order from big to small, and K is a positive integer;

And obtaining the equipment voiceprint vector corresponding to the equipment audio data according to the characteristic data and the equipment state parameter.

4. The method of claim 3, wherein the step of obtaining the device voiceprint vector corresponding to the device audio data from the feature data and the device state parameter comprises:

and inputting the characteristic data and the equipment state parameters into a neural network after splicing to obtain the equipment voiceprint vector corresponding to the equipment audio data.

5. The method of claim 3, wherein the step of obtaining the device voiceprint vector corresponding to the device audio data from the feature data and the device state parameter comprises:

the characteristic data and the equipment state parameters are spliced and then input into a first part of a neural network to obtain network hidden layer characteristics, wherein the neural network comprises the first part and a second part;

and after the equipment state parameters and the network hidden layer characteristics are spliced, inputting the equipment state parameters and the network hidden layer characteristics into the second part of the neural network to obtain the equipment voiceprint vector corresponding to the equipment audio data.

6. The method according to any one of claims 1-5, wherein the step of determining the fault type of the target device from the correspondence in the target fault sample library comprises:

Obtaining a second similarity set according to the similarity of the fault voiceprint vector and each fault voiceprint vector in the target fault sample library;

if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold, determining a target fault type corresponding to a fault voiceprint vector corresponding to the maximum similarity in the target fault sample library according to the corresponding relation in the target fault sample library, wherein the target fault type is the fault type of the target equipment.

7. The method of claim 6, wherein the step of determining the type of failure of the target device based on the correspondence in the target failure sample library further comprises:

if the maximum similarity in the second similarity set is smaller than the second preset threshold, outputting prompt information, wherein the prompt information is used for prompting the input of the fault type;

in response to obtaining an input fault type, determining that the fault type of the target device is the input fault type;

the method further comprises the steps of:

and writing the corresponding relation between the fault voiceprint vector and the input fault type into the target fault sample library.

8. A fault type determining apparatus, comprising:

the first determining module is used for determining the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices normally operate;

the classification module is used for classifying the different devices of the same type into a plurality of subtypes according to the similarity; wherein the similarity between different devices in one of the sub-types is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library comprises a corresponding relation between fault voiceprint vectors and fault types;

the acquisition module is used for acquiring the equipment voiceprint vectors of the target equipment in the different equipment of the same type;

the second determining module is used for determining that the equipment voiceprint vector of the target equipment is a fault voiceprint vector, and determining the fault type of the target equipment according to the corresponding relation in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target equipment belongs.

9. A computer device, the device comprising: a processor and a memory;

The memory is used for storing program codes by a computer and transmitting the computer program codes to the processor;

the processor is configured to perform the method of determining a fault type according to any of claims 1-7 according to instructions in the computer program code.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when run on the device of claim 9, causes the device to perform the method of determining a fault type according to any of claims 1-7.