WO2023096185A1

WO2023096185A1 - Method for diagnosing machine failure on basis of deep-learning by using sounds and vibrations, and diagnosis device using same

Info

Publication number: WO2023096185A1
Application number: PCT/KR2022/016624
Authority: WO
Inventors: 김병희; 이충연
Original assignee: 주식회사 써로마인드
Priority date: 2021-11-29
Filing date: 2022-10-27
Publication date: 2023-06-01
Also published as: KR20230080242A

Abstract

The present invention pertains to a method for diagnosing machine failure on the basis of deep-learning by using sounds or vibrations. The method comprises the steps in which: (a) when sound data or vibration data detected from a machine is acquired, a diagnosis device inputs the sound data or the vibration data to a feature extraction unit so that the feature extraction unit extracts features 1 to m from the sound data or the vibration data; (b) the diagnosis device inputs features 1 to m to deep-learning models 1 to n so as to cause deep-learning models 1 to n to learn features 1 to m and output failure classification information 1_1 to n_m classifying failure states for features 1 to m; and (c) the diagnosis device outputs failure diagnosis information diagnosing the failure of the machine by ensembling failure classification information 1_1 to n_m.

Description

Method for diagnosing machine failure using sound and vibration based on deep learning and diagnostic device using the same

The present invention relates to a method for diagnosing a failure of a machine, and more particularly, to a method for diagnosing a failure of a machine using sound and vibration based on deep learning and a diagnostic device using the same.

In general, mechanical facilities that must be operated continuously in industrial sites can stop equipment lines due to sudden failures that cause fatal damage to the entire equipment line and unexpected failures, and machine failures to prevent the resulting decrease in production. Methods for predicting in advance are being researched and developed.

A currently used failure prediction method detects an abnormal state of a machine, detects a defect that appears at this time, determines the state of a machine or a mechanical part, and predicts whether or not a failure occurs.

On the other hand, due to the recent development of deep learning networks, a method for diagnosing a failure state of a machine using a deep learning network has been proposed.

However, since the characteristics of data for each machine and each process are different according to the failure state, an individual diagnosis algorithm reflecting the characteristics of each data is required to effectively diagnose the failure of the machine. However, it takes a lot of time and money to build individual diagnostic algorithms, and since the built diagnostic algorithm is applied only to the corresponding data, there is a problem of low efficiency.

In addition, when using an individual diagnosis algorithm in which each data characteristic is reflected, there is a problem in that failure types other than the learned failure types cannot be accurately determined.

Therefore, the present invention intends to propose a method for diagnosing machine failures based on deep learning regardless of the type of data.

The present invention has as its object to solve all the problems mentioned above.

Another object of the present invention is to diagnose machine failures based on deep learning regardless of the type of input data.

In addition, another object of the present invention is to accurately diagnose a failure of a machine using sound data or vibration data based on deep learning.

In addition, another object of the present invention is to actively cope with unlearned failure types by using sound data or vibration data based on deep learning.

According to one embodiment of the present invention for achieving the above object, in a method for diagnosing a failure of a machine using sound or vibration based on deep learning, (a) when sound data or vibration data detected from a machine are obtained , inputting the sound data or the vibration data to a feature extraction unit, and causing the feature extraction unit to extract first to m-th features from the sound data or the vibration data; (b) the diagnosis device inputs the first feature to the m-th feature as a first deep learning model to an n-th deep learning model, and causes the first to n-th deep learning models to perform the first performing a running operation on features to the m-th feature to output 1_1 th failure classification information to n_m th failure classification information obtained by classifying failure states for the first feature to the m-th feature; and (c) outputting, by the diagnostic device, failure diagnosis information for diagnosing a failure of the machine by ensembling the 1st to 1st failure classification information to the n_mth failure classification information; A method including is provided.

In the step (b), the diagnosis device concatenates the first feature to the m-th feature to generate an m-channel feature map having each of the first feature to the m-th feature as a channel; The m-channel feature maps may be input to the first to n-th deep learning models.

The first to n-th deep learning models may include at least one supervised learning model and at least one unsupervised learning model for classifying the failure state of the machine.

The diagnosis device may update a learning parameter of the unsupervised learning model using the failure diagnosis information.

The sound data or the vibration data is generated by sampling the sound or vibration generated by the machine in a predetermined time unit, and when the length of the sample is longer than the predetermined time unit, the data in the previous predetermined time unit Only sampling is performed, and when the length of the sample is shorter than the predetermined time unit, zero padding may be added to the rear part.

The first feature to the mth feature include a Mel spectrogram divided by a frequency range in consideration of human hearing, an MFCC that is a coefficient when the Mel spectrogram is regarded as one signal and cosine-transformed, and a signal sign for a unit time Zero crossing rate, which is the number of transitions, spectral Roll-Off, which is the value of f when the energy corresponding to the frequency below f matches a specific ratio of the total energy, spectral centroid, which is the center of gravity or weighted average of the spectrum, and the variance of the spectral distribution It may include at least two of spectral bandwidth, spectral contrast, which is a difference between maximum and minimum energies in a partial spectrum, spectral flatness, which is a flatness of a spectrum, and chromagram, which is a harmonic characteristic of a spectrum.

In the step (c), the diagnosis apparatus inputs the 1_1-th failure classification information to the n_m-th failure classification information to a discriminator for determining whether a failure exists, and the 1_1-th failure classification information to the n_m-th failure classification information Determination information for each failure is output, and the 1_1-th failure classification information to the n_m-th failure classification information and the determination information are input to the model evaluation unit so that the model evaluation unit can perform the 1_1 failure classification information through to the n_m-th failure classification information. The failure diagnosis information, which diagnoses the failure of the machine by ensembling the n_m-th failure classification information, may be output.

In the step (c), the diagnosis device determines the failure of the machine among the 1_1 th failure classification information to the n_m th failure classification information by referring to the probability of answering each of the 1_1 th failure classification information to the n_m th failure classification information. It is possible to select specific failure classification information for , and output the specific failure classification information as the failure diagnosis information.

In the step (c), the diagnosis apparatus may output the failure diagnosis information by performing a weighted sum of the 1_1 th failure classification information to the n_m th failure classification information.

In addition, according to an embodiment of the present invention, in the diagnostic apparatus for diagnosing machine failures using sound or vibration based on deep learning, instructions for diagnosing machine failures using sound or vibration based on deep learning memory in which they are stored; and a processor configured to perform an operation for diagnosing a failure of a machine using sound or vibration based on the deep learning according to the instructions stored in the memory. The processor includes: (i) when the sound data or vibration data detected from the machine is obtained, inputs the sound data or the vibration data to a feature extractor so that the feature extractor can obtain the sound data or the vibration data A process of extracting first to m-th features, (ii) inputting the first to m-th features to a first to n-th deep learning model to extract the first to n-th deep learning models; A process of causing a deep learning model to perform a running operation on the first feature to the m-th feature and output 1_1 th failure classification information to n_m th failure classification information obtained by classifying failure states for the first feature to m th feature. , and (iii) outputting fault diagnosis information for diagnosing a fault of the machine by ensembling the 1_1th fault classification information to the n_m-th fault classification information.

The processor, in the process (ii), concatenates the first feature to the m-th feature to generate an m-channel feature map having each of the first feature to the m-th feature as a channel, An m-channel feature map may be input to the first to n-th deep learning models.

The processor may update learning parameters of the unsupervised learning model using the failure diagnosis information.

In the (iii) process, the processor inputs the 1_1 th failure classification information to the n_m th failure classification information to a discriminator for determining whether a failure exists, and the 1_1 th failure classification information to the n_m th failure classification information, respectively. outputs determination information for determining whether there is a failure, and inputs the 1_1th failure classification information to the n_m-th failure classification information and the determination information to the model evaluation unit so that the model evaluation unit can perform the 1_1 failure classification information to the above The failure diagnosis information in which the failure of the machine is diagnosed by ensembling the n_m-th failure classification information may be output.

In the (iii) process, the processor determines the failure of the machine among the 1_1 th failure classification information to the n_m th failure classification information by referring to a probability value of an answer of each of the 1_1 th failure classification information to the n_m th failure classification information. Specific failure classification information may be selected and the specific failure classification information may be output as the failure diagnosis information.

In the process (iii), the processor may output the failure diagnosis information by performing a weighted sum of the 1_1 th failure classification information to the n_m th failure classification information.

In addition to this, a computer readable recording medium for recording a computer program for executing the method of the present invention is further provided.

According to the present invention, there are the following effects.

The present invention makes it possible to diagnose machine failures based on deep learning regardless of the type of input data.

According to the present invention, machine failure can be accurately diagnosed using sound data or vibration data based on deep learning.

The present invention can actively cope with unlearned failure types by using sound data or vibration data based on deep learning.

1 schematically illustrates a diagnostic device for diagnosing a failure of a machine using sound or vibration based on deep learning according to an embodiment of the present invention;

2 is a flow chart schematically showing a method for diagnosing a machine failure using sound or vibration based on deep learning according to an embodiment of the present invention;

3 is a block diagram schematically showing a method for diagnosing a machine failure using sound or vibration based on deep learning according to an embodiment of the present invention;

4 schematically illustrates features extracted from sound data or vibration data in a method for diagnosing machine failure using sound and vibration based on deep learning according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable one skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the present invention.

1 schematically illustrates a diagnosis apparatus for diagnosing a failure of a machine using sound or vibration based on deep learning according to an embodiment of the present invention.

Referring to FIG. 1 , a diagnosis apparatus 1000 according to an embodiment of the present invention includes a memory 1001 in which instructions for diagnosing machine failures based on deep learning using sound or vibration are stored, and the memory 1001 It may include a processor 1002 that performs an operation for diagnosing a failure of a machine using sound or vibration based on deep learning according to stored instructions.

Specifically, diagnostic device 1000 is typically a computing device (e.g., a device that may include computer processors, memory, storage, input and output devices, and other components of conventional computing devices; electronic devices such as routers, switches, and the like). A desired system using a combination of communication devices; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (ie, instructions that enable a computing device in a particular way) It may be to achieve performance.

In addition, the processor of the computing device may include hardware components such as a micro processing unit (MPU) or a central processing unit (CPU), a cache memory, and a data bus. In addition, the computing device may further include an operating system and a software component of an application performing a specific purpose.

However, it is not excluded that the computing device includes an integrated processor in which a medium for implementing the present invention, a processor, and a memory are integrated.

Meanwhile, when the sound data or vibration data detected from the machine is obtained, the processor 1002 of the diagnosis apparatus 1000 inputs the sound data or vibration data to the feature extraction unit according to instructions stored in the memory 1001 to the feature extraction unit. extracts first to m-th features from the acoustic data or vibration data, and inputs the first to m-th features to the first to n-th deep learning models to obtain the first to n-th deep learning models; The running model performs a running operation on the first to m-th features and outputs 1_1-th failure classification information to n_m-th failure classification information in which failure states for the first to m-th features are classified, and 1_1 failure classification information It is possible to output fault diagnosis information in which faults of the machine are diagnosed by ensembling the through n_m-th fault classification information.

A method of diagnosing a machine failure using sound or vibration based on deep learning using the diagnostic device 1000 according to an embodiment of the present invention configured as described above will be described with reference to FIGS. 2 and 3 as follows. .

First, when sound data or vibration data detected from a machine is acquired, the diagnosis apparatus 1000 inputs the sound data or vibration data to the feature extractor 100 so that the feature extractor 100 can obtain the sound data or vibration data. The first to mth features may be extracted from (S10). m may be an integer of 2 or greater.

In this case, the sound data or vibration data may be generated by sampling the sound or vibration generated by the machine in a predetermined unit of time.

In addition, the sound data or vibration data may be a spectrum of sound or vibration generated in the machine, directly acquired and sampled from a sensor installed in the machine, or obtained by sampling data sensed from the sensor by other equipment. can

On the other hand, if the length of the sample of sound data or vibration data obtained from the machine is longer than the preset time unit, only the data of the preset time unit of the front portion is sampled, and if the length of the sample is shorter than the preset time unit, the second portion is sampled. You can add zero padding to the part.

In addition, the diagnosis apparatus 1000 selects various types of features, that is, first to mth features, from sound data or vibration data through the feature extractor 100 in order to detect various failure types related to machine failures. can be extracted.

For example, referring to FIG. 4 , the first to mth features may be features of different types extracted from one piece of data, such as acoustic data or vibration data, and, as shown in FIG. A feature corresponding to the Mel spectrogram obtained by dividing the frequency range in consideration of , a feature corresponding to MFCC, which is a coefficient when the Mel spectrogram is considered as one signal and cosine-transformed as shown in (b) of FIG. As in ), the feature corresponding to the zero crossing rate, which is the number of times the sign of the signal changes during unit time, as in FIG. A feature corresponding to spectral roll-off, which is the value of f, a feature corresponding to the spectral centroid, which is the center of gravity of the spectrum or a weighted average, as shown in (e) of FIG. A feature corresponding to spectral bandwidth, a feature corresponding to spectral contrast, which is the difference between maximum and minimum energy in a partial spectrum, as shown in FIG. 4 (g), and spectral flatness, which is the flatness of the spectrum, Corresponding features, as shown in (i) of FIG. 4 , may include at least two features among features corresponding to chromagrams, which are harmonic characteristics of a spectrum.

Next, the diagnosis apparatus 1000 inputs the first to m-th features to the first to n-th deep learning models, and causes the first to n-th deep learning models to output the first to m-th features. Fault classification information 1_1th to n_mth failure classification information obtained by classifying the failure states of the first to mth features by running the features may be output ( S20 ). n may be an integer of 2 or greater.

In this case, the first to n-th deep learning models may include at least one supervised learning model and at least one unsupervised learning model for classifying the failure state of the machine. Also, the first to n-th deep learning models may include at least one reinforcement learning model for classifying the failure state of the machine.

In addition, supervised learning models include KNN (K-Nearest Nejghbor), Native Bayes, Support Vector, Machine Decision, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Convolutional Recurrent Neural Network (CRNN), and AutoEncoder. It can include application models and regression models such as Linear Regression, Locally Weighted Linear, Ridge, and Lasso. Unsupervised learning models include Clustering, K-Means, Density Estimation, Exception Maximization, Pazen Window, DBSCAN, and LSTM (Long- Short Term Memory) AutoEncoder and the like. However, the present invention is limited to this may include various models for diagnosing machine failures by classifying features.

Meanwhile, the diagnosis apparatus 1000 inputs the first feature to each of the first to n-th deep learning models, and each of the first to n-th deep learning models performs a running-running operation on the first feature to obtain 1_1 To output failure classification information to n_1-th failure classification information, and input the second feature to each of the first deep learning model to the n-th deep learning model, so that each of the first deep learning model to the n-th deep learning model uses the second feature The running operation is performed to output 1_2 th failure classification information to n_2 th failure classification information, which is repeated, and inputs the m th feature to the first deep learning model to the n th deep learning model, respectively, to obtain the first deep learning model to the n_th Each of the n deep learning models performs a running operation on the m-th feature and outputs 1_m-th failure classification information to n_m-th failure classification information, thereby generating 1_1-th failure classification information to n_m-th failure classification information.

Also, unlike this, the diagnostic apparatus 1000 concatenates the first to m-th features to generate an m-channel feature map having each of the first to m-th features as channels, and the m-channel feature map It is possible to generate 1_1 th failure classification information to n_m th failure classification information by inputting the data to the first to nth deep learning models.

That is, the diagnosis apparatus 1000 inputs the m-channel feature map to the first deep learning model and causes the first deep learning model to perform a running operation on the m-channel feature map and output 1_1 failure classification information to 1_m failure classification information. The m-channel feature map is input to the second deep learning model, and the second deep learning model performs a running operation on the m-channel feature map to output 2_1 failure classification information to 2_m failure classification information, and repeating this, The m-channel feature map is input to the n-th deep learning model, and the n-th deep learning model performs a running operation on the m-channel feature map to output n_1-th failure classification information to n_m-th failure classification information, thereby obtaining 1_1 failure classification information to The n_mth failure classification information may be generated.

Next, the diagnosis apparatus 1000 may ensemble the 1_1 th failure classification information to the n_m th failure classification information to output failure diagnosis information for diagnosing a failure of the machine (S30).

For example, the failure diagnosis apparatus 1000 inputs 1_1 th failure classification information to n_m th failure classification information to a discriminator 300 that determines whether or not there is a failure, and inputs 1_1 th failure classification information to n_m th failure classification information. Determination information for each failure is outputted, and the 1_1 to n_m failure classification information and determination information are input to the model evaluation unit 400 so that the model evaluation unit 400 can classify the 1_1 failure. Information through n_m-th failure classification information may be ensembled to output failure diagnosis information for diagnosing a failure of a machine.

At this time, the diagnostic apparatus 1000 refers to the probability of answering each of the 1_1th failure classification information to the n_mth failure classification information, and the specific failure classification information for the failure of the machine among the 1_1th failure classification information to the n_mth failure classification information. can be selected, and specific failure classification information can be output as failure diagnosis information.

For example, by referring to the correct answer probability value of each of the 1_1st fixed classification information to the n_m-th failure classification information, selecting specific failure classification information having the highest probability of correct answer value, or corresponding to a specific failure type having the highest average correct answer probability value for each failure type. Failure diagnosis information may be output by ensembling 1st_1st fixed classification information to n_mth failure classification information by various methods, such as selecting specific failure classification information.

In addition, the diagnosis apparatus 1000 may weight-sum the 1st to 1st failure classification information to the n_mth failure classification information to output failure diagnosis information.

In addition, the diagnosis apparatus 1000 may classify the 1_1th to n_m-th failure classification information by failure type, and output specific failure classification information corresponding to the failure type with the highest frequency as failure diagnosis information. .

However, the present invention is not limited thereto, and the failure diagnosis information may be output by ensembling the 1_1th failure classification information to the n_mth failure classification information in various ways.

In addition, the diagnosis apparatus 1000 updates learning parameters of unsupervised learning models among the first deep learning model to the n-th deep learning model using the failure diagnosis information, thereby actively responding to failure types of machines that were not aired at the time of initial design. be able to cope with

In addition, the diagnosis apparatus 1000 continuously learns the first to n-th deep learning models using the failure diagnosis information, thereby diagnosing failures of the first to n-th deep learning models, respectively. Accuracy can be improved.

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

In the above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art to which the present invention pertains may seek various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the spirit of the present invention. will do it

Claims

A method for diagnosing machine failure using sound or vibration based on deep learning,

(a) When the sound data or vibration data detected from the machine is obtained, the diagnosis device inputs the sound data or the vibration data to a feature extractor to cause the feature extractor to obtain first features to first features from the sound data or the vibration data. allowing an mth feature to be extracted;

(b) the diagnosis device inputs the first feature to the m-th feature as a first deep learning model to an n-th deep learning model, and causes the first to n-th deep learning models to perform the first performing a running operation on features to the m-th feature to output 1_1 th failure classification information to n_m th failure classification information obtained by classifying failure states for the first feature to the m-th feature; and

(c) outputting, by the diagnosis device, failure diagnosis information for diagnosing a failure of the machine by ensembling the 1st failure classification information to the n_m-th failure classification information;

How to include.
According to claim 1,

In step (b),

The diagnosis apparatus concatenates the first to m-th features to generate m-channel feature maps including each of the first to m-th features as channels, and converts the m-channel feature maps into the m-th feature maps. A method of inputting input to the first deep learning model to the nth deep learning model.
According to claim 1,

The first deep learning model to the n-th deep learning model include at least one supervised learning model and at least one unsupervised learning model for classifying the failure state of the machine.
According to claim 3,

The diagnosis device updates learning parameters of the unsupervised learning model using the failure diagnosis information.
According to claim 1,

The sound data or the vibration data is generated by sampling the sound or vibration generated by the machine in a predetermined time unit, and when the length of the sample is longer than the predetermined time unit, the data in the previous predetermined time unit A method of sampling only, and adding zero padding to a rear part when the length of the sample is shorter than the preset time unit.
According to claim 1,

The first feature to the mth feature include a Mel spectrogram divided by a frequency range in consideration of human hearing, an MFCC that is a coefficient when the Mel spectrogram is regarded as one signal and cosine-transformed, and a signal sign for a unit time Zero crossing rate, which is the number of transitions, spectral Roll-Off, which is the value of f when the energy corresponding to the frequency below f matches a specific ratio of the total energy, spectral centroid, which is the center of gravity or weighted average of the spectrum, and the variance of the spectral distribution A method that includes at least two of spectral bandwidth, spectral contrast, which is the difference between maximum and minimum energy in a partial spectrum, spectral flatness, which is the flatness of a spectrum, and chromagram, which is a harmonic characteristic of a spectrum.
According to claim 1,

In step (c),

The diagnosis apparatus determines whether each of the 1_1-th failure classification information to the n_m-th failure classification information has a failure by inputting the 1_1-th failure classification information to the n_m-th failure classification information into a discriminator that determines whether or not there is a failure. The determination information is output, and the 1_1 th failure classification information to the n_m th failure classification information and the determination information are input to a model evaluation unit so that the model evaluation unit determines the 1_1 th failure classification information to the n_m th failure classification information. A method of ensembling and outputting the failure diagnosis information for diagnosing a failure of the machine.
According to claim 1,

In step (c),

The diagnostic apparatus obtains specific failure classification information for the failure of the machine from among the 1_1th failure classification information to the n_mth failure classification information with reference to a correct answer probability value of each of the 1_1th failure classification information to the n_mth failure classification information. and outputting the specific failure classification information as the failure diagnosis information.
According to claim 1,

In step (c),

The method of claim 1 , wherein the diagnosis apparatus weights and sums the 1st to 1st failure classification information to the n_mth failure classification information to output the failure diagnosis information.
A diagnostic device for diagnosing machine failure using sound or vibration based on deep learning,

a memory in which instructions for diagnosing a failure of a machine using sound or vibration based on deep learning are stored; and

a processor configured to perform an operation for diagnosing a failure of a machine using sound or vibration based on the deep learning according to the instructions stored in the memory;

Including,

The processor may (i) when the sound data or vibration data detected from the machine is obtained, input the sound data or the vibration data to a feature extractor, and cause the feature extractor to obtain first features to first features from the sound data or the vibration data. a process of extracting the m-th feature, (ii) inputting the first feature to the m-th feature to a first deep learning model to an n-th deep learning model into the first deep learning model to the n-th deep learning model; a process of performing a running operation on the first feature to the m-th feature and outputting 1_1 th failure classification information to n_m th failure classification information obtained by classifying the failure states for the first feature to the m-th feature; and (iii) ) The diagnosis apparatus performs a process of outputting fault diagnosis information in which a fault of the machine is diagnosed by ensembling the 1_1th fault classification information to the n_m-th fault classification information.
According to claim 10,

The processor, in the process (ii), concatenates the first feature to the m-th feature to generate an m-channel feature map having each of the first feature to the m-th feature as a channel, A diagnostic device for inputting m-channel feature maps to the first to n-th deep learning models.
According to claim 10,

The first deep learning model to the n-th deep learning model include at least one supervised learning model and at least one unsupervised learning model for classifying the failure state of the machine.
According to claim 12,

The processor updates the learning parameters of the unsupervised learning model using the failure diagnosis information.
According to claim 10,

The sound data or the vibration data is generated by sampling the sound or vibration generated by the machine in a predetermined time unit, and when the length of the sample is longer than the predetermined time unit, the data in the previous predetermined time unit A diagnostic device that samples only the samples and adds zero padding to a rear part when the length of the sample is shorter than the preset time unit.
According to claim 10,

The first feature to the mth feature include a Mel spectrogram divided by a frequency range in consideration of human hearing, an MFCC that is a coefficient when the Mel spectrogram is regarded as one signal and cosine-transformed, and a signal sign for a unit time Zero crossing rate, which is the number of transitions, spectral Roll-Off, which is the value of f when the energy corresponding to the frequency below f matches a specific ratio of the total energy, spectral centroid, which is the center of gravity or weighted average of the spectrum, and the variance of the spectral distribution A diagnostic device that includes at least two of spectral bandwidth, spectral contrast, which is the difference between maximum and minimum energy in a partial spectrum, spectral flatness, which is the flatness of a spectrum, and chromagram, which is a harmonic characteristic of a spectrum.
According to claim 10,

In the (iii) process, the processor inputs the 1_1 th failure classification information to the n_m th failure classification information to a discriminator for determining whether a failure exists, and the 1_1 th failure classification information to the n_m th failure classification information, respectively. outputs determination information for determining whether there is a failure, and inputs the 1_1th failure classification information to the n_m-th failure classification information and the determination information to the model evaluation unit so that the model evaluation unit can perform the 1_1 failure classification information to the above A diagnosis device configured to output failure diagnosis information obtained by ensembling n_m-th failure classification information to diagnose failures of the machine.
According to claim 10,

In the (iii) process, the processor determines the failure of the machine among the 1_1 th failure classification information to the n_m th failure classification information by referring to a probability value of an answer of each of the 1_1 th failure classification information to the n_m th failure classification information. A diagnostic device for selecting specific failure classification information for the failure classification and outputting the specific failure classification information as the failure diagnosis information.
According to claim 10,

wherein the processor outputs the failure diagnosis information by weighting the 1_1 th failure classification information to the n_m th failure classification information in the (iii) process.