JP7304301B2 - Acoustic diagnostic method, acoustic diagnostic system, and acoustic diagnostic program - Google Patents

Description

The present invention relates to an acoustic diagnostic method, an acoustic diagnostic system, and an acoustic diagnostic program.

Equipment abnormalities and signs of failure often appear in sounds. Acoustic diagnosis based on the operation sound of equipment is therefore important in order to grasp the condition of the equipment. Acoustic diagnosis requires a feature extraction method that embeds the essential features of operating sounds.

As a method of acoustically diagnosing the state of equipment, Patent Document 1 describes, "Elevator diagnosis system obtains feature amounts from measurement data (sensor data), and determines the operating state of the elevator based on the measurement data of the elevator to be diagnosed. select the combination of the normal model and diagnostic threshold corresponding to the combination of the type or identifier of the elevator to be diagnosed and the operating state specified for the elevator, and obtain from the measurement data acquired for the elevator to be diagnosed Based on the feature value and the normal model, the probability that the feature value is observed in the normal state is obtained, and the state of the elevator is diagnosed by comparing the obtained probability with the threshold obtained for the elevator to be diagnosed.Elevator diagnosis system , for example, learns a plurality of normal models with different degrees of complexity, and selects the one with the lowest complexity among the plurality of trained normal models.”

JP 2018-95429 A

The technique disclosed in the above-mentioned Patent Literature 1 extracts a feature amount from the signal itself measured by the acoustic sensor and inputs it into a learned normal model, thereby estimating the state of the facility. However, in this prior art, when the frequency structure of the target equipment sound changes over time, or when the environmental noise is large, the feature amount space becomes complicated. There is a problem that it can not be done.

The present invention has been made in consideration of the above points, and one of the objects of the present invention is to enable accurate state estimation without requiring a large amount of learning data in acoustic diagnosis for diagnosing the state of equipment. aim.

In order to solve the above problems, in one aspect of the present invention, a sound diagnosis method executed by a sound diagnosis system includes an input sound acquisition step of acquiring an input sound including an operation sound of equipment to be diagnosed; a source separation step of separating the spectrogram into a harmonic sound spectrogram and a sudden sound spectrogram by harmonic sound-sudden sound source separation; and concatenating the harmonic sound spectrogram and the sudden sound spectrogram. The method includes a feature amount vector generation step of generating a feature amount vector, and a state estimation step of estimating the state of the facility to be diagnosed based on the feature amount vector and the learning model.

According to the present invention, for example, in acoustic diagnosis for diagnosing the state of equipment, it is possible to accurately estimate the state without requiring a large amount of learning data.

FIG. 3 is a block diagram showing the configuration of the state estimation model of the acoustic diagnosis system according to Embodiment 1 during learning; 5 is a flow chart showing processing during learning of the state estimation model of the acoustic diagnosis system according to the first embodiment. 2 is a block diagram showing the configuration of the acoustic diagnostic system according to Embodiment 1 when state estimation is executed; FIG. 5 is a flow chart showing processing during state estimation execution of the acoustic diagnostic system according to the first embodiment. FIG. 11 is a block diagram showing the configuration of the acoustic diagnostic system according to Embodiment 2 during learning of a normal sound model; FIG. 11 is a block diagram showing the configuration of the acoustic diagnosis system according to the second embodiment when abnormality detection is executed; FIG. 12 is a block diagram showing the configuration of the state estimation model of the acoustic diagnosis system according to Embodiment 3 during learning; 10 is a flowchart showing processing during learning of a state estimation model of the acoustic diagnosis system according to Embodiment 3; FIG. 12 is a block diagram showing the configuration of the acoustic diagnostic system according to the third embodiment when state estimation is executed; 11 is a flow chart showing processing during state estimation execution of the acoustic diagnostic system according to the third embodiment. FIG. 12 is a block diagram showing the configuration of the state estimation model of the acoustic diagnosis system according to the fourth embodiment during learning; FIG. 12 is a flowchart showing processing during learning of the state estimation model of the acoustic diagnosis system according to the fourth embodiment; FIG. FIG. 12 is a block diagram showing the configuration of the acoustic diagnostic system according to the fourth embodiment when state estimation is executed; FIG. 16 is a flow chart showing processing during execution of state estimation of the acoustic diagnosis system according to the fourth embodiment; FIG. FIG. 12 is a block diagram showing the configuration of the state estimation model of the acoustic diagnosis system according to Embodiment 5 during learning; FIG. 14 is a block diagram showing the configuration of the acoustic diagnostic system according to Embodiment 5 when state estimation is executed; It is a figure which shows the hardware constitutions of the computer which implement|achieves an acoustic diagnostic system.

Preferred embodiments of the present invention are described below. In the following, the same or similar elements and processes are denoted by the same reference numerals, and overlapping descriptions are omitted. Further, in the embodiments described later, only differences from the previously described embodiments will be described, and redundant description will be omitted.

In addition, the configuration and processing shown in the following description and drawings are intended to illustrate the outline of the embodiments to the extent necessary for understanding and implementing the present invention, and are intended to limit the embodiments according to the present invention. not intended to do so. Moreover, each embodiment and each modification can be combined in whole or in part without departing from the gist of the present invention and within a matching range.

[Embodiment 1]
<Configuration during learning of the state estimation model of the acoustic diagnostic system 1 of the first embodiment>
FIG. 1 is a block diagram showing the configuration during learning of the state estimation model of the acoustic diagnosis system 1 according to the first embodiment. The acoustic diagnosis system 1 includes an input sound acquisition unit 11, a preprocessing unit 12, a harmonic sound-sudden sound source separation unit 13, a vector connection unit 14, a state estimation model learning unit 15, and a It has a state estimation model database 16 .

The input sound acquisition unit 11 converts an analog input sound including an operation sound of equipment for acoustic diagnosis acquired or recorded via a microphone into a digital input sound (time domain digital input sound).

The preprocessing unit 12 divides the digital input sound converted by the input sound acquisition unit 11 into frames, multiplies the frames by a window function, performs Fourier transform on the window function multiplied signal, and obtains a frequency domain signal. to calculate In calculating the frequency domain signal, a Fast Fourier Transform (FFT), a Short-Time Fourier Transform (STFT), or other frequency analysis techniques may be used.

The frequency domain signal calculated by the preprocessor 12 is a set of M complex numbers, one complex number corresponding to each of (N/2+1)=M frequency bins, if the frame size is N. . Furthermore, the preprocessing unit 12 calculates an input sound spectrogram (power spectrogram or amplitude spectrogram) from the frequency domain signal.

Harmonic/Percussive Sound Separation (HPSS) section 13 separates the input sound spectrogram calculated by preprocessing section 12 into harmonic sound components and sudden sound components. As HPSS, there are those using median filters (Fitzgerald, D. (2010). Harmonic/Percussive Separation using Median Filtering. 13th International Conference on Digital Audio Effects (DAFX10), Graz, Austria, 2010.) and spectrogram time Based on changes (Hideyuki Tachibana, Nobutaka Ono, Shigeki Sagayama.(2009). Enhancement and suppression of singing voice components from music audio signals based on spectral temporal changes. Research Report Music Information Science (MUS), 2009(12) ), 1-6.), based on anisotropy of spectrogram smoothness (Tachibana, H., Ono, N., Kameoka, H., & Sagayama, S. (2014). Harmonic/percussive sound separation based on anisotropic smoothness of spectrograms. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(12), 2059-2073.) may be used. For example, when using HPSS based on the time change of the spectrogram, the objective function is defined as Equation (1), and the constraint is defined as Equation (2).

Here, the spectrograms of the input signal, harmonic components, and sudden components are denoted by W _t,k , H _t,k , and P _{t,k ,} respectively, where t and k represent the time index and frequency index, respectively.

The above optimization problem can be approximated by iteratively obtaining equations (3) and (4).

The vector connecting unit 14 connects a vector of harmonic sound components (harmonic sound spectrogram) and a vector of sudden sound components (sudden sound spectrogram) to generate a feature quantity vector. The state estimation model learning unit 15 performs model learning based on the plurality of feature amount vectors obtained by the vector connection unit 14 and stores the learned state estimation model in the state estimation model database 16 .

Support Vector Classifier (SVC), 1-Class Support Vector Classifier, Multi-Class Support Vector Classifier, Hidden Markov Model (HMM), Nearest Neighbor Classifier, etc. may be used as the state estimation model.

<Learning processing of the state estimation model of the first embodiment>
FIG. 2 is a flowchart showing processing during learning of the state estimation model of the acoustic diagnosis system 1 according to the first embodiment.

In step S11, the input sound acquisition unit 11 converts analog input sounds for learning, including operating sounds of the facility to be acoustically diagnosed, which are acquired or recorded via a microphone, into digital input sounds (time-domain digital input sounds). Convert.

Next, in step S12, the preprocessing unit 12 divides the digital input sound converted by the input sound acquiring unit 11 into frames. Next, in step S13, the preprocessing unit 12 multiplies the frame divided in step S12 by a window function. Next, in step S14, the preprocessing unit 12 performs Fourier transform on the signal after multiplication with the window function in step S13 to calculate a frequency domain signal. Next, in step S15, the preprocessing unit 12 calculates an input sound spectrogram from the frequency domain signal calculated in step S14.

Next, in step S16, the harmonic sound-sudden sound source separation unit 13 separates the input sound spectrogram calculated by the preprocessing unit 12 into a harmonic sound spectrogram and a sudden sound spectrogram. Next, in step S17, the vector connecting unit 14 generates a feature amount vector obtained by vector connecting the harmonic sound spectrogram and the sudden sound spectrogram. Next, in step S<b>18 , the state estimation model learning unit 15 learns a state estimation model based on the feature amount vector obtained by the vector connection unit 14 and stores it in the state estimation model database 16 .

<Configuration of Acoustic Diagnosis System 1 of Embodiment 1 When State Estimation is Executed>
FIG. 3 is a block diagram showing the configuration of the acoustic diagnostic system 1 according to the first embodiment when state estimation is executed. The acoustic diagnostic system 1 includes an input sound acquisition unit 11, a preprocessing unit 12, a harmonic sound-sudden sound source separation unit 13, a vector connection unit 14, a state estimation model database 16, a state estimation unit 21, as a configuration for executing state estimation. , and a state estimation result output unit 22 .

The state estimating unit 21 reads a state estimating model from the state estimating model database 16 and executes state estimating processing with the feature amount vector generated by the vector connecting unit 14 as an input. That is, the state estimating unit 21 calculates the time series of the feature vector consisting of L consecutive frames, and calculates the probability of generating the time series for each state class estimated from the feature vector. .

The state estimation result output unit 22 outputs the estimation result by the state estimation unit 21 . For example, the state estimation result output unit 22 may output each state class and the corresponding probability, or may output the state class with the maximum probability.

<State Estimation Execution Processing of Embodiment 1>
FIG. 4 is a flow chart showing the processing during state estimation execution of the acoustic diagnosis system 1 according to the first embodiment. In this process, in step S11, the input sound acquisition unit 11 acquires or records analog input sounds for diagnosis including operating sounds of equipment for acoustic diagnosis, which are acquired or recorded via a microphone, as digital input sounds (time domain digital input sound).

Further, in step S21 subsequent to step S17, the state estimation unit 21 inputs the feature amount vector generated by the vector connection unit 14 based on the state estimation model read out from the state estimation model database 16, and performs state estimation processing. Execute. Next, in step S22, the state estimation result output unit 22 outputs the state estimation result estimated in step S21.

<Effect of Embodiment 1>
According to this embodiment, since it can be assumed that artificial equipment sounds such as machines essentially belong to either harmonic sound components or sudden sound components, by using harmonic sound-sudden sound source separation (HPSS) , it is possible to obtain effective feature amounts for state estimation even when the target sound is complex.

In addition, when the characteristics of the target sound differ depending on the state of the target (for example, the distribution of the harmonic sound component and the sudden sound component changes so that the sound that is steady in the normal state changes to a non-stationary sound in the abnormal state) , etc.), by combining the vectors of the harmonic sound component and the sudden sound component separated by HPSS, it is possible to obtain a feature amount that can cope with changes, and to improve the accuracy of abnormal sound detection.

[Embodiment 2]
<Configuration during learning of the state estimation model of the acoustic diagnostic system 1B of the second embodiment>
FIG. 5 is a block diagram showing the configuration of the acoustic diagnosis system 1B according to the second embodiment during normal sound model learning. The acoustic diagnosis system 1B is an anomaly detection system, and differs from the first embodiment in that the state of equipment estimated by acoustic diagnosis is limited to two, normal or abnormal. For this reason, compared to the acoustic diagnostic system 1, the acoustic diagnostic system 1B has a normal sound model learning configuration instead of the state estimation model learning unit 15 and the state estimation model database 16 as a configuration for learning a normal sound model. It has a section 15B and a normal sound model database 16B.

The normal sound model learning unit 15B performs model learning of the normal distribution of the feature amount vectors composed of consecutive L frames based on the plurality of feature amount vectors obtained by the vector connecting unit 14, and generates the learned normal sound model. is stored in the normal sound model database 16B.

As normal sound models, Gaussian mixture distribution (GMM), one-class support vector classifier, subspace method, local subspace method, k-means clustering, Deep Neural Network (DNN) autoencoder, Convolutional Neural Network (CNN) autoencoder , Long Short Term Memory (LSTM) autoencoder, variational autoencoder (VAE), etc. may be used.

Algorithms suitable for each normal sound model are known and used for learning. For example, in the case of GMM, an EM algorithm is used to apply a combination of Gaussian distributions corresponding to a predetermined number of clusters. A learned normal sound model is defined by the calculated model parameters. All the model parameters are stored in the normal sound model database 16B.

<Configuration of Acoustic Diagnosis System 1B of Embodiment 2 at State Estimation Execution>
FIG. 6 is a block diagram showing the configuration of the acoustic diagnosis system 1B according to the second embodiment when abnormality detection is executed. Compared to the acoustic diagnostic system 1, the acoustic diagnostic system 1B has an abnormality detection unit instead of the state estimation unit 21, the state estimation result output unit 22, and the state estimation model database 16 as a configuration for executing state estimation. 21B, an anomaly level output unit 22B, and a normal sound model database 16B.

The abnormality detection unit 21B reads a normal sound model from the normal sound model database 16B, and executes state estimation processing on the spectrogram to be diagnosed. That is, it calculates a time series of feature amount vectors consisting of L consecutive frames, and determines whether or not the time series can be generated from the normal sound model with a sufficient probability.

For example, when the normal sound model is GMM, the abnormality detection unit 21B detects that the M×L-dimensional feature amount vector v is the normal sound model (model parameters Θ=((μ1, Γ1, π1), . . . (μq, Γq , πq), (μQ, ΓQ, πQ)) is calculated by equations (5) and (6).

In this case, the anomaly detection unit 21B outputs, for example, the negative logarithmic likelihood “-log(p(v|Θ))” of probability p(v|Θ) as the estimated anomaly degree.

In addition, when a Deep Neural Network (DNN) autoencoder is used as a normal sound model, the anomaly detection unit 21B uses an optimization algorithm such as SGD, Momentum SGD, AdaGrad, RMSprop, AdaDelta, Adam, etc., to perform input normal The internal parameters are optimized such that the restoration error between the sound feature amount vector and the output feature amount vector is less than a predetermined value. Therefore, when a feature amount vector of an abnormal sound is input, it is expected that the restoration error between the input feature amount vector of the abnormal sound and the output feature amount vector is greater than or equal to a predetermined value. Therefore, the abnormality detection unit 21B outputs this restoration error as the estimated degree of abnormality.

The degree-of-abnormality output unit 22B outputs the value of the estimated degree of abnormality and, if the value of the estimated degree of abnormality is equal to or higher than a certain value, outputs that there is an abnormality.

In the process of learning the state estimation model of the acoustic diagnostic system 1B of the second embodiment, in the process of learning the state estimation model of the acoustic diagnostic system 1 of the first embodiment (FIG. 2), in step S18, normal sound The model learning unit 15B learns a normal sound model based on the feature vector obtained by the vector connecting unit 14, and stores the normal sound model in the normal sound model database 16B.

Further, in the process of executing the state estimation of the acoustic diagnostic system 1B of the second embodiment, in the process of executing the state estimation of the acoustic diagnostic system 1 of the first embodiment (FIG. 4), in step S21, the abnormality detection unit 21B Based on the state estimation model read out from the normal sound model database 16B, the feature amount vector generated by the vector connecting unit 14 is input, and the abnormality detection process is executed. Also, in step S22, the abnormality degree output unit 22B outputs the abnormality detection result estimated in step S21B.

<Effect of Embodiment 2>
According to the present embodiment, it is possible to determine whether a sound to be diagnosed is a normal sound or an abnormal sound.

[Embodiment 3]
<Configuration during learning of the state estimation model of the acoustic diagnostic system 1C of the third embodiment>
FIG. 7 is a block diagram showing the configuration during learning of the state estimation model of the acoustic diagnostic system 1C according to the third embodiment. This embodiment aims to improve the state estimation accuracy by removing low-frequency components as noise. Compared to the first embodiment, the acoustic diagnosis system 1C has a nearest neighborhood filtering unit 12C in front of the harmonic sound-sudden sound source separation unit 13 as a configuration for learning the state estimation model, and filters low frequency components ( The difference is that sudden or irregular sounds such as voices, work sounds, running water sounds, etc., are removed as unnecessary noise.

The nearest neighbor filtering unit 12C applies a nearest neighbor filter to the input sound spectrogram output by the preprocessing unit 12, separates it into a low frequency component and a high frequency component, and generates a low frequency component-removed spectrogram obtained by removing the low frequency component. Output. The harmonic sound-sudden sound source separation unit 13 separates the low frequency component-removed spectrogram from which the low frequency components have been removed by the nearest neighborhood filtering unit 12C into a harmonic sound spectrogram and a sudden sound spectrogram.

<Learning processing of the state estimation model of the third embodiment>
FIG. 8 is a flowchart showing processing during learning of the state estimation model of the acoustic diagnosis system 1C according to the third embodiment. The processing during learning of the state estimation model of the acoustic diagnostic system 1C according to the third embodiment differs from the processing during learning of the state estimation model of the acoustic diagnostic system 1 according to the first embodiment (FIG. 2) in the following points. is different.

That is, in step S15C subsequent to step S15, the nearest neighbor filtering unit 12C applies a nearest neighbor filter to the input sound spectrogram output by the preprocessing unit 12, separates it into low frequency components and high frequency components, Output the spectrogram after removing the low-frequency components. In step S16 following step S15C, the harmonic sound-sudden sound source separation unit 13 separates the low frequency component-removed spectrogram from which the low frequency components have been removed by the nearest neighbor filtering unit 12C into a harmonic sound spectrogram and a sudden sound spectrogram.

<Configuration of Acoustic Diagnosis System 1C of Embodiment 3 During State Estimation Execution>
FIG. 9 is a block diagram showing the configuration of the acoustic diagnostic system 1C according to the third embodiment when state estimation is executed. The acoustic diagnostic system 1C differs from the acoustic diagnostic system 1 in that it has a nearest neighbor filtering section 12C in front of the harmonic sound-sudden sound source separation section 13 as a configuration for executing state estimation.

<Processing when estimating the state of the acoustic diagnostic system 1C of the third embodiment>
FIG. 10 is a flow chart showing the processing during state estimation execution of the acoustic diagnostic system 1C according to the third embodiment. The state estimation execution process of the acoustic diagnostic system 1C according to the third embodiment differs from the state estimation execution process (FIG. 4) of the acoustic diagnostic system 1 according to the first embodiment in the following points.

That is, in step S15C, the nearest neighbor filtering unit 12C outputs a low-frequency-component-removed spectrogram obtained by removing low-frequency components from the input sound spectrogram calculated by the preprocessing unit 12. FIG. Next, in step S16, the harmonic sound-sudden sound source separation unit 13 separates the spectrogram after the low-frequency component removal output by the nearest neighbor filtering unit 12C into a harmonic sound spectrogram and a sudden sound spectrogram.

<Effect of Embodiment 3>
According to this embodiment, for example, low-frequency components (sudden or irregular sounds such as voices, work sounds, and running water sounds) that can be assumed to be environmental noises that are not likely to be diagnostic target devices are filtered by the NN filter in the preceding stage of the HPSS. By removing it, the complexity of the feature amount of the target sound can be reduced, and the accuracy of acoustic diagnosis based on the feature amount can be improved.

[Embodiment 4]
In the present embodiment, speech data acquired or recorded via microphones of multiple channels is used, and the state estimation accuracy is enhanced by filtering without distortion. Compared with Embodiment 3, this embodiment is different in that the data used for diagnosis is extended from a single channel to multiple channels, and distortion-free filtering is realized.

<Configuration during learning of the state estimation model of the acoustic diagnostic system 1D of the fourth embodiment>
FIG. 11 is a block diagram showing the configuration during learning of the state estimation model of the acoustic diagnostic system 1D according to the fourth embodiment.

The acoustic diagnostic system 1D includes a multi-channel input sound acquisition unit 11D, a preprocessing unit 12, a nearest neighbor filtering unit 12C, a harmonic sound-sudden sound source separation unit 13, and a steering vector generation unit 13D1 as a configuration for learning the state estimation model. , 13D6, noise generation units 13D2, 13D7, spatial covariance matrix calculation units 13D3, 13D8, filter calculation units 13D4, 13D9, filtering units 13D5, 13D10, vector connection unit 14, state estimation model learning unit 15, and state estimation model database 16.

The multi-channel input sound acquisition unit 11D converts analog input sounds recorded by multi-channel microphones into digital input sounds.

The steering vector generator 13D1 generates a steering vector when the harmonic sound spectrogram is the target sound. Also, the steering vector generation unit 13D6 generates a steering vector when the sudden sound spectrogram is used as the target sound. Although the steering vector generators 13D1 and 13D6 are shown as different configurations in FIG. 11, they may have a single configuration.

The noise generation unit 13D2 mixes the sudden sound spectrogram separated by the harmonic sound-sudden sound source separation unit 13 and the low frequency component spectrogram separated by the nearest neighbor filtering unit 12C to generate noise. The noise generation unit 13D7 also mixes the harmonic sound spectrogram separated by the harmonic sound-sudden sound source separation unit 13 and the low-frequency component spectrogram separated by the nearest neighbor filtering unit 12C to generate noise. Although the noise generators 13D2 and 13D7 are shown as different configurations in FIG. 11, they may have a single configuration.

The spatial covariance matrix calculator 13D3 obtains the spatial covariance matrix of the noise generated by the noise generator 13D2. Also, the spatial covariance matrix calculator 13D8 obtains the spatial covariance matrix of the noise generated by the noise generator 13D7. Spatial covariance matrix calculators 13D3 and 13D8 are shown as different configurations in FIG. 11, but may have a single configuration.

The filter calculator 13D4 obtains a filter for emphasizing the harmonic sound from the steering vector with the harmonic sound as the target sound and the spatial covariance matrix with the sudden sound as noise. The filter calculator 13D9 obtains a filter for emphasizing the sudden sound from the steering vector with the sudden sound as the target sound and the spatial covariance matrix with the harmonic sound as noise. Minimum Variance Distortionless Response (MVDR) or the like may be used for filter calculation. Although the filter calculation units 13D4 and 13D9 are shown as different configurations in FIG. 11, they may have a single configuration.

The filtering unit 13D5 applies the filter for emphasizing the harmonic sound calculated by the filter calculating unit 13D4 to the input sound obtained by the multichannel input sound obtaining unit 11D, and obtains undistorted harmonic sound components. Further, the filtering unit 13D10 applies the filter for emphasizing the sudden sound calculated by the filter calculating unit 13D9 to the input sound acquired by the multi-channel input sound acquiring unit 11D, and obtains undistorted sudden sound components. Although the filtering units 13D5 and 13D10 are shown as different configurations in FIG. 11, they may have a single configuration.

The vector connecting unit 14 connects the undistorted harmonic sound component calculated by the filtering unit 13D5 and the undistorted sudden sound component calculated by the filtering unit 13D10.

<Processing during learning of the state estimation model of the acoustic diagnosis system 1D of the fourth embodiment>
FIG. 12 is a flowchart showing processing during learning of the state estimation model of the acoustic diagnosis system 1D according to the fourth embodiment. The processing during learning of the state estimation model of the acoustic diagnostic system 1D according to the fourth embodiment differs from the processing during learning of the state estimation model of the acoustic diagnostic system 1C according to the third embodiment (FIG. 8) in the following points. is different.

In step S11D, the multi-channel input sound acquisition unit 11D acquires or records analog input sounds for learning, including operating sounds of the facility to be acoustically diagnosed, acquired or recorded via the multi-channel microphones as digital input sounds (time-domain digital input sound).

In step S16D1 following step S16, the steering vector generator 13D1 uses the harmonic sound spectrogram to generate a steering vector when the harmonic sound spectrogram is the target sound. Also, in step S16D1, the steering vector generator 13D6 uses the sudden sound spectrogram to generate a steering vector when the sudden sound spectrogram is used as the target sound.

Further, in step S16D2 following step S16, the noise generator 13D2 mixes the sudden sound spectrogram and the low frequency component spectrogram to generate noise, and the spatial covariance matrix calculator 13D3 generates noise generated by the noise generator 13D2. Find the spatial covariance matrix. In step S16D2, the noise generator 13D7 mixes the harmonic sound spectrogram and the low-frequency component spectrogram to generate noise, and the spatial covariance matrix calculator 13D8 generates the spatial covariance matrix of the noise generated by the noise generator 13D7. Ask for

Next, in step S16D3, the filter calculator 13D4 obtains a filter that emphasizes harmonics from the steering vector generated by the steering vector generator 13D1 and the spatial covariance matrix calculated by the spatial covariance matrix calculator 13D3. Also, in step S16D3, the filter calculator 13D9 obtains a filter that emphasizes the sudden sound from the steering vector generated by the steering vector generator 13D6 and the spatial covariance matrix calculated by the spatial covariance matrix calculator 13D8.

Next, in step S16D4, the filtering unit 13D5 applies the filter for emphasizing the harmonic sound calculated by the filter calculating unit 13D4 to the input sound from the multichannel input sound acquiring unit 11D, thereby obtaining undistorted harmonic sound. ask for ingredients. Further, in step S16D4, the filtering unit 13D10 applies the filter for emphasizing the sudden sound calculated by the filter calculating unit 13D9 to the input sound from the multi-channel input sound acquiring unit 11D, thereby obtaining an undistorted sudden sound. ask for ingredients.

Next, in step S17, the vector connecting unit 14 vector-connects the undistorted harmonic sound component and the undistorted sudden sound component obtained in step S16D4.

<Configuration of Acoustic Diagnosis System 1D of Embodiment 4 When State Estimation is Executed>
FIG. 13 is a block diagram showing the configuration of the acoustic diagnostic system 1D according to the fourth embodiment when state estimation is executed. Compared to the configuration for acoustic model learning, the acoustic diagnostic system 1D includes a state estimation unit 21, a state estimation unit 21, and a state estimation model database 16 instead of the state estimation model learning unit 15 and the state estimation model database 16. It has a result output unit 22 and a state estimation model database 16 .

<Processing when estimating the state of the acoustic diagnostic system 1D of the fourth embodiment>
FIG. 14 is a flow chart showing processing when state estimation is executed in the acoustic diagnosis system according to the fourth embodiment. The processing of the acoustic diagnosis system 1D according to the fourth embodiment when performing state estimation is different from the processing when learning the state estimation model (FIG. 12) in that steps S21 and S22 are executed instead of step S18. different.

<Effect of Embodiment 4>
In this embodiment, the target sound including the operating sound of the facility for acoustic diagnosis acquired or recorded using a multi-channel microphone is separated into a high frequency component and a low frequency component, and the high frequency component is an H component (harmonic sound component) and P component (sudden sound component). A steering vector for the H component is generated, and the spatial covariance matrix of the noise for the H component is determined using the P component and the low frequency component. Also, a P-component steering vector is generated, and a spatial covariance matrix of noise for the P-component is obtained using the H component and the low-frequency component. Then, by using these steering vectors and spatial covariance matrices to generate a filter that emphasizes the target sound and perform filtering, it is possible to improve the accuracy of abnormality detection.

[Embodiment 5]
In this embodiment, a filter for emphasizing either the harmonic sound or the sudden sound whose sound source is separated by the HPSS is calculated, and steering is performed from either the harmonic sound or the sudden sound for the multi-channel target sound. A vector is generated and a spatial covariance matrix is generated from the other, and filtering is performed to emphasize either the harmonic sound or the burst sound to improve the state estimation accuracy.

This embodiment differs from the fourth embodiment in that the target sound is limited to either one of the harmonic sound and the sudden sound, and the vectors of the spectrogram after filtering are not concatenated. In particular, this embodiment is effective when it is clear whether the target sound is a harmonic sound or a sudden sound, and whether the target sound is a harmonic sound or a sudden sound does not change regardless of the state of the equipment. is.

<Configuration during learning of the state estimation model of the acoustic diagnostic system 1E of the fifth embodiment>
FIG. 15 is a block diagram showing the configuration during learning of the state estimation model of the acoustic diagnostic system 1E according to the fifth embodiment.

The acoustic diagnostic system 1E includes a multi-channel input sound acquisition unit 11D, a preprocessing unit 12, a nearest neighbor filtering unit 12C, a harmonic sound-sudden sound source separation unit 13, and a steering vector generation unit 13E1 as a configuration for learning the state estimation model. , a spatial covariance matrix calculation unit 13E2, a filter calculation unit 13E3, a filtering unit 13E4, a state estimation model learning unit 15, and a state estimation model database 16.

The steering vector generation unit 13E1 uses the harmonic sound spectrogram obtained by separating the input sound by the harmonic sound-sudden sound source separation unit 13 to generate a steering vector when the harmonic sound spectrogram is used as the target sound. The spatial covariance matrix calculator 13E2 calculates a spatial covariance matrix from the sudden sound spectrogram.

The filter calculator 13E3 obtains a filter for emphasizing the harmonic sound from the steering vector with the harmonic sound as the target sound and the spatial covariance matrix with the sudden sound as noise. The filtering unit 13E4 applies the filter for emphasizing the harmonic sound calculated by the filter calculating unit 13E3 to the input sound obtained by the multichannel input sound obtaining unit 11D, and obtains undistorted harmonic sound components.

<Configuration of Acoustic Diagnosis System 1E of Embodiment 5 When State Estimation is Executed>
FIG. 16 is a block diagram showing the configuration of the acoustic diagnostic system 1E according to the fifth embodiment when state estimation is executed. Compared to the configuration for acoustic model learning, the acoustic diagnostic system 1E includes a state estimating unit 21 and a state estimation result output instead of the state estimating model learning unit 15 and the state estimating model database 16 as the configuration for executing state estimation, compared to the configuration for acoustic model learning. It has a part 22 and a state estimation model database 16 .

In FIGS. 15 and 16, it is assumed that the steering vector is generated from the harmonic sound spectrogram, the spatial covariance matrix is calculated from the sudden sound spectrogram, and undistorted harmonic sound components are obtained by emphasizing the harmonic sound of the target sound. there is However, the present invention is not limited to this, and a steering vector may be generated from the sudden sound spectrogram, a spatial covariance matrix may be calculated from the harmonic sound spectrogram, and undistorted sudden sound components emphasizing the sudden sound of the target sound may be obtained from these.

It should be noted that in the processing during learning of the state estimation model of the acoustic diagnostic system 1E of the fifth embodiment, the processing during learning of the state estimation model of the acoustic diagnostic system 1D of the fourth embodiment (FIG. 12) and the processing during execution of state estimation ( 14), in step S16D1, the steering vector generator 16D1 generates a steering vector from the harmonic sound spectrogram separated by the harmonic sound-sudden sound source separator 13. In FIG. Further, in step S16D2, the spatial covariance matrix calculator 13E2 calculates a spatial covariance matrix from the sudden sound spectrogram separated by the harmonic sound-sudden sound source separator 13. FIG.

<Effect of Embodiment 5>
In this embodiment, when the target sound and the ambient noise are separated into harmonic sound components and sudden sound components, respectively, the harmonic sound components are used to generate the steering vector of the target sound, and the sudden sound components are used to generate the spatial coherence of the noise. A variance matrix can be obtained. By using them to generate a filter that emphasizes the target sound and filtering, it is possible to improve the accuracy of anomaly detection.

<Computer 5000 realizing acoustic diagnostic systems 1, 1B, 1C, 1D, and 1E>
FIG. 17 is a diagram showing the hardware configuration of a computer 5000 that implements the acoustic diagnostic systems 1, 1B, 1C, 1D and 1E.

A computer 5000 that implements the acoustic diagnostic systems 1, 1B, 1C, 1D, and 1E includes a processor 5300 represented by a CPU (Central Processing Unit), a memory 5400 such as a RAM (Random Access Memory), an input device 5600 (for example, a single channel connection interfaces such as microphones, multi-channel microphones, keyboards, mice, touch panels, etc.) and output devices 5700 (eg, a video graphics card connected to an external display monitor) are interconnected through memory controller 5500 . In the computer 5000, a predetermined program is read from an external storage device 5800 such as an SSD or HDD via an I/O (Input/Output) controller 5200, and executed by cooperation of a processor 5300 and a memory 5400. , an acoustic diagnostic system is realized. Alternatively, the program for implementing the acoustic diagnostic system may be acquired from an external computer through communication via network interface 5100. FIG. Also, the program for realizing the acoustic diagnostic system may be recorded on a recording medium and read and acquired by a medium reading device.

The present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, as long as there is no contradiction, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, replace, integrate, or distribute a part of the configuration of each embodiment. Also, the configurations and processes shown in the embodiments can be appropriately distributed, integrated, or replaced based on processing efficiency or implementation efficiency.

1, 1B, 1C, 1D, 1E: acoustic diagnosis system, 11: input sound acquisition unit, 11D: multi-channel input sound acquisition unit, 12: preprocessing unit, 12C: nearest neighbor filtering unit, 13: harmonic sound-sudden sound Sound source separation unit 13D1, 13D6, 13E1: steering vector generation unit 13D2, 13D7: noise generation unit 13D3, 13D8, 13E2: spatial covariance matrix calculation unit 13D4, 13D9, 13E3: filter calculation unit 13D5, 13D10, 13E4: filtering unit, 14: vector connecting unit, 15: state estimation model learning unit, 15B: normal sound model learning unit, 16: state estimation model database, 16B: normal sound model database, 21: state estimation unit, 21B: abnormality Detection unit, 22: state estimation result output unit, 22B: abnormality degree output unit, 5000: computer

Claims (12)

An acoustic diagnostic method performed by an acoustic diagnostic system, comprising:
an input sound acquisition step of acquiring an input sound including the operation sound of the equipment to be diagnosed;
a preprocessing step of calculating a spectrogram of the input sound;
a sound source separation step of separating the spectrogram into a harmonic sound spectrogram and a sudden sound spectrogram by harmonic sound-sudden sound source separation;
a feature vector generation step of generating a feature vector connecting the harmonic sound spectrogram and the sudden sound spectrogram;
and a state estimation step of estimating the state of the facility to be diagnosed based on the feature quantity vector and the learning model. The acoustic diagnosis method according to claim 1, further comprising a state estimation result outputting step of outputting the state of the facility to be diagnosed estimated by the state estimating step. 2. The acoustic diagnostic method according to claim 1, further comprising: a learning model generating step of generating the learning model based on the feature vector generated from the input sound for learning. The learning model is a normal sound model obtained by performing model learning of the normal distribution of the feature vector generated from the input sound for learning,
4. The apparatus according to claim 3, wherein in the state estimation step, it is estimated whether or not the facility to be diagnosed is normal based on the feature vector generated from the input sound for diagnosis and the learning model. Acoustic diagnostic method. a frequency separation step of separating the spectrogram calculated by the preprocessing step into a high frequency component spectrogram and a low frequency component spectrogram;
5. The sound source separation step, wherein the spectrogram from which the low-frequency component spectrogram is removed is separated into the harmonic sound spectrogram and the sudden sound spectrogram in the sound source separation step. or 1. The acoustic diagnostic method according to 1. acquiring the multi-channel input sound in the input sound acquiring step;
a frequency separation step of separating the spectrogram calculated by the preprocessing step into a high-frequency component spectrogram and a low-frequency component spectrogram, and inputting the high-frequency component spectrogram to the sound source separation step;
Of the harmonic sound spectrogram and the sudden sound spectrogram from which the high-frequency component spectrogram is separated by the sound source separation step, generating a first steering vector from the harmonic sound spectrogram and generating a second steering vector from the sudden sound spectrogram. a steering vector generation step for generating
a spatial covariance matrix calculation step of calculating a first spatial covariance matrix from the sudden sound spectrogram and the low frequency component spectrogram, and calculating a second spatial covariance matrix from the harmonic sound spectrogram and the low frequency component spectrogram; ,
calculating a first filter from the first steering vector and the first spatial covariance matrix, and calculating a second filter from the second steering vector and the second spatial covariance matrix; and,
Obtaining the undistorted harmonic sound spectrogram by applying the first filter to the multi-channel input sound obtained by the input sound obtaining step, and applying the second filter to the input sound. a filtering step of obtaining said sudden sound spectrogram without distortion by
including
5. The feature vector generating step generates the feature vector by connecting the harmonic sound spectrogram and the sudden sound spectrogram obtained by the filtering step. The acoustic diagnostic method described in . An acoustic diagnostic method performed by an acoustic diagnostic system,
an input sound acquisition step of acquiring multi-channel input sounds including operating sounds of equipment to be diagnosed;
a preprocessing step of calculating a spectrogram of the input sound;
a frequency separation step of separating the spectrogram into a high frequency component spectrogram and a low frequency component spectrogram;
A sound source separation step of separating the post-removal spectrogram obtained by removing the low-frequency component spectrogram from the spectrogram into a harmonic sound spectrogram and a sudden sound spectrogram by harmonic sound-sudden sound source separation;
generating a steering vector from a first one of the harmonic sound spectrogram and the sudden sound spectrogram;
a spatial covariance matrix calculation step of calculating a spatial covariance matrix from a second one of the harmonic sound spectrogram and the sudden sound spectrogram;
a filter calculation step of calculating a filter from the steering vector and the spatial covariance matrix;
a filtering step of obtaining the undistorted first spectrogram by applying the filter to the multi-channel input sound obtained by the input sound obtaining step;
A sound diagnosis method, comprising: a state estimation step of estimating a state of the facility to be diagnosed based on the undistorted first spectrogram obtained by the filtering step and a learning model. The acoustic diagnosis method according to claim 7, further comprising a state estimation result outputting step of outputting the state of the facility to be diagnosed estimated by the state estimating step. 8. The acoustic diagnostic method according to claim 7, further comprising a learning model generation step of generating the learning model based on the undistorted first spectrogram generated from the input sound for learning. An acoustic diagnostic system for diagnosing the state of equipment to be diagnosed,
an input sound acquisition unit that acquires input sounds including operation sounds of equipment to be diagnosed;
a preprocessing unit that calculates a spectrogram of the input sound;
a sound source separation unit that separates the spectrogram into a harmonic sound spectrogram and a sudden sound spectrogram by harmonic sound-sudden sound source separation;
a feature vector generation unit that generates a feature vector connecting the harmonic sound spectrogram and the sudden sound spectrogram;
A sound diagnosis system, comprising: a state estimating unit that estimates a state of the facility to be diagnosed based on the feature quantity vector and the learning model. An acoustic diagnostic system for diagnosing the state of equipment to be diagnosed,
an input sound acquisition unit that acquires multi-channel input sounds including operating sounds of equipment to be diagnosed;
a preprocessing unit that calculates a spectrogram of the input sound;
a frequency separator that separates the spectrogram into a high frequency component spectrogram and a low frequency component spectrogram;
a sound source separation unit that separates the post-removal spectrogram obtained by removing the low-frequency component spectrogram from the spectrogram into a harmonic sound spectrogram and a sudden sound spectrogram by harmonic sound-sudden sound source separation;
a steering vector generator that generates a steering vector from a first spectrogram of the harmonic sound spectrogram and the sudden sound spectrogram;
a spatial covariance matrix calculator that calculates a spatial covariance matrix from a second one of the harmonic sound spectrogram and the sudden sound spectrogram;
a filter calculator that calculates a filter from the steering vector and the spatial covariance matrix;
a filtering unit that obtains the undistorted first spectrogram by applying the filter to the multi-channel input sound acquired by the input sound acquisition unit;
A sound diagnosis system, comprising: a state estimating section for estimating a state of the facility to be diagnosed based on the undistorted first spectrogram obtained by the filtering section and a learning model. An acoustic diagnostic program for causing a computer to function as the acoustic diagnostic system according to claim 10 or 11.

Images