JP6142402B2 - Acoustic signal analyzing apparatus, method, and program - Google Patents

Description

  The present invention relates to an acoustic signal analysis device, method, and program, and more particularly, to an acoustic signal analysis device, method, and program for enhancing an original voice signal from time-series data of the acoustic signal.

  The voice enhancement technique is an important element technique for improving the intelligibility of communication voice in a hands-free call system and improving the voice recognition rate in an automatic conference record creation system, for example.

  Since information on noise and reverberation environment is often not known in advance, it is necessary to deal with the problem of acquiring a speech signal from only the observation signal, but the process by which reverberation and noise are superimposed on the speech signal to obtain the observation signal Can be regarded as an inverse problem.

  If the information on the noise and the indoor transmission system is unknown, there can be countless solutions to this inverse problem, so some assumption is necessary to narrow down the solution. In conventional approaches related to speech enhancement, there are many cases where the inverse problem is formulated under the assumptions of constraints such as stationarity for noise and time invariance for indoor transmission systems.

  For example, in the spectral subtraction method, which is a classical noise removal technique, noise continuity is assumed, and the noise power (or amplitude) spectrum obtained from a known non-speech interval is derived from the power (or amplitude) spectrum of the observed signal. The power (or amplitude) spectrum of the audio signal is estimated by subtraction. In addition, the dereverberation method based on inverse filtering assumes a time-invariant and minimum-phase indoor transmission system, and is best suited to clues to assumptions and models related to speech (harmonics, autoregressive models, autocorrelation function codebooks, etc.) Estimate the dereverberation filter.

  While these conventional methods exhibit high performance in the situation where the assumption is established, in the real environment, the noise may be non-stationary, such as a ringtone of a mobile phone and background music, or slightly However, when there is a movement of the sound source, the above assumptions about noise and the indoor transmission system may not be satisfied, and there is a problem that high performance cannot be exhibited under such an environment.

  As a robust dereverberation method against changes in the indoor transmission system, a method has been proposed that estimates the speech spectrogram based on the temporal convolution model of the speech spectrogram and the room impulse response spectrogram based on the sparsity of the speech spectrogram. (Non-Patent Document 1).

  However, in this method, only a multiplicative noise mixing process such as reverberation is assumed, and an additive noise mixing process is not assumed explicitly (assuming a situation where no noise exists). When the observation signal contains noise, there is a problem that high performance cannot be exhibited.

  On the other hand, as a technique for suppressing non-stationary noise, a non-negative matrix factorization approach that considers a spectrogram as a non-negative matrix and decomposes it into two non-negative matrices has recently attracted attention (Non-Patent Document 2). . If non-negative matrix factorization is used, the base spectrum constituting the observation spectrogram and their coupling coefficients at each time can be obtained simultaneously. As a result, for example, when a base spectrum is acquired (learned) in advance from a clean speech sample, and the observation spectrogram of a noisy speech is decomposed into a base matrix and a coefficient matrix, a base spectrum obtained by learning a part of the base spectrum set in advance. It is decomposed into a spectrogram of speech and other (ie noise).

H. Kameoka, T. Nakatani, T. Yoshioka, “Robust Speech Dereverberation Based on Non-negativity and Sparse Nature of Speech Spectrograms,” in Proc. 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2009), pp. 45 −48, 2009. P. Smaragdis, B. Raj and M. V. Shashanka, “Supervised and semi-supervised separation of sounds from single-channel mixture,” in Proc. ICA2007, pp. 414-421, 2007.

  However, in the above-mentioned method of Non-Patent Document 2, only an additive noise mixing process is assumed, and a multiplicative noise mixing process such as reverberation is not assumed explicitly (assuming a situation without reverberation). Therefore, there is a problem that high performance cannot be exhibited when reverberation is included in the observed signal.

  The present invention has been made in view of the above circumstances, and emphasizes an original audio signal by accurately separating the original audio signal from time-series data of an acoustic signal in which noise and a reverberation component are superimposed on the original audio signal. An object of the present invention is to provide an acoustic signal analyzing apparatus, method, and program capable of performing the above.

In order to achieve the above object, the acoustic signal analyzing apparatus according to the present invention receives time series data of an acoustic signal as an input and outputs an observation time frequency component y _k [t] of each frequency k at each time t. The analysis unit and the power spectrum density Φ ^(s) _k [t] at each time t and each frequency k of the original speech signal s included in the acoustic signal are each non-negative element B representing M base spectra m. ^(s) _m, and the base matrix of _k, as the product of the basis spectra B ^(s) _{m, k} of the gain G ^(s) _m [t] a coefficient matrix for each element is a non-negative value at each time t Each element G ^(s) _m [t] of the coefficient matrix when expressed, power H _k [τ] of each frequency k of the indoor impulse response at each time τ, and each time of the noise signal n included in the acoustic signal The power spectral density Φ ⁽ⁿ⁾ _k [t] of t and each frequency k is determined from non-negative elements B ⁽ⁿ⁾ _{q, k} representing the Q base spectra q. And a base matrix B ⁽ⁿ⁾ _{q, k} at each time t is expressed as a product of a coefficient matrix having each non-negative gain G ⁽ⁿ⁾ _q [t] Each element B ⁽ⁿ⁾ _{q, k} , and each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix are set to initial values, and each element B ^(s) _{m, k} of the base matrix is set to A parameter initial value setting unit for setting values obtained in advance and elements B ^(s) _{m, k of the} base matrix and elements G ^(s) _m [of the coefficient matrix in all combinations of (τ, m) t], the power H _k [τ], and the power spectral density series model Φ ^x _k [t] calculated based on the power spectral density Φ ⁽ⁿ⁾ _k [t], and the observation time frequency component the observation time frequency component y _k [t] of each time t and each frequency k, each element B ^(s) _{m, k of the} basis matrix, so that the distance to y _k [t] becomes small, Coefficient matrix An element _{^{G (s) m [t]}} , the elements B ⁽ⁿ⁾ _q of the basis _matrix, and _k, and the elements G ⁽ⁿ⁾ _q [t] of the coefficient matrix, the room impulse response for each time τ Based on the power H _k [τ] at each frequency k and the power spectral density series model Φ ^x _k [t] at each time t and each frequency k, each element G ^(s) _m [t] of the coefficient matrix And each element B ⁽ⁿ⁾ _{q, k of the} basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and power H _k [τ of each frequency k of the room impulse response at each time τ And a termination determination unit that repeatedly performs updating by the parameter updating unit until a predetermined termination condition is satisfied.

In the acoustic signal analysis method according to the present invention, the time-frequency analysis unit inputs the time-series data of the acoustic signal, outputs the observation time frequency component y _k [t] of each frequency k at each time t, and sets the parameter initial value The setting unit converts the power spectrum density Φ ^(s) _k [t] at each time t and each frequency k of the original audio signal s included in the acoustic signal into each non-negative element B representing M base spectra m. ^(s) _m, and the base matrix of _k, as the product of the basis spectra B ^(s) _{m, k} of the gain G ^(s) _m [t] a coefficient matrix for each element is a non-negative value at each time t Each element G ^(s) _m [t] of the coefficient matrix when expressed, power H _k [τ] of each frequency k of the indoor impulse response at each time τ, and each time of the noise signal n included in the acoustic signal power spectral density Φ of t and each frequency k a ⁽ⁿ⁾ _k [t], the elements of the non-negative value representing the number Q of the base spectrum q ⁽ⁿ⁾ _q, and the base matrix of _k, as the product of the basis spectra B ⁽ⁿ⁾ _{q, k} of the gain G ⁽ⁿ⁾ _q [t] a coefficient matrix for each element is a non-negative value at each time t The initial value of each element B ⁽ⁿ⁾ _{q, k of} the base matrix and the element G ⁽ⁿ⁾ _q [t] of the coefficient matrix is set, and each element B ^{( s)} A value obtained in advance is set in _{m, k} , and the parameter updating unit performs element B ^(s) _{m, k of the} base matrix and elements of the coefficient matrix in all combinations of (τ, m). _{^{G (s) m [t]}} , and with the power H _k [τ], the power spectral density Φ ⁽ⁿ⁾ _k [t] power spectral density series model is calculated based on the Φ ^x _k [t] and such that said distance between the observation time-frequency component y _k [t] is small, and the observation time-frequency component y _k [t] at each time t and each frequency k, each element of the basis matrix B ^(s) _m, and _k, before Each element G of the coefficient matrix ^(s) _m [t], wherein the elements B ⁽ⁿ⁾ _q basis _matrix, and _k, each element G of the coefficient matrix ⁽ⁿ⁾ _q [t], at each time τ Based on the power H _k [τ] of each frequency k of the indoor impulse response and the power spectral density series model Φ ^x _k [t] of each time t and each frequency k, each element G ^{(s) of the} coefficient matrix _m [t], each element B ⁽ⁿ⁾ _{q, k of the} basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and power of each frequency k of the indoor impulse response at each time τ H _k [τ] is updated, and the updating by the parameter updating unit is repeated until the end determination unit satisfies a predetermined end condition.

  The program according to the present invention is a program for causing a computer to function as each part of the acoustic signal analysis apparatus.

As described above, according to the acoustic signal analysis apparatus, method, and program of the present invention, the base matrix and coefficient matrix of the power spectral density Φ ^(s) _k [t] of the original audio signal s, and the room impulse response The power spectral density series model Φ ^x _k [t] calculated based on the power H _k [τ] and the power spectral density Φ ⁽ⁿ⁾ _k [t] of the noise signal n, and the observation time frequency component y _k [ as the distance between t] is small, the coefficient matrix of the power spectral density _{^{Φ (s) k [t]}} , and the base matrix and the coefficient matrix of the power spectral density _{^{Φ (n) k [t]}} , the room impulse response By repeatedly updating the power H _k [τ], the original voice signal is accurately separated from the time series data of the acoustic signal in which noise and reverberation components are superimposed on the original voice signal, and the original voice signal is emphasized. The effect of being able to be obtained.

It is a figure which shows the amplitude component and phase component of short-time Fourier-transform of the indoor impulse response measured by changing the position of a microphone. It is the schematic which shows the structure of the acoustic signal analyzer which concerns on embodiment of this invention. It is a flowchart which shows the content of the acoustic signal analysis process routine in the acoustic signal analyzer which concerns on embodiment of this invention. (A) A graph showing the evaluation result of the Mel frequency cepstrum coefficient distortion, (B) a graph showing the evaluation result of the Bark spectrum distortion score, and (C) a graph showing the evaluation result of the signal-to-interference ratio.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Principle of the invention>

  First, the principle of the present invention will be described.

<Formulation of speech enhancement problem>

  When the indoor transmission system is time-invariant, if the original audio signal is s [t], the indoor impulse response is h [t], and the noise signal is n [t], the observation signal y [t] as an acoustic signal is the time domain Is expressed as the following equation (1).

Where t is an index of discrete time. The first term on the right side _Στ s [t−τ] h [τ] represents a reverberant speech signal. Here, y _k [t], s _k [t], h _k [t], and n _k [t] are the short-time Fourier transforms of the observed signal, the original signal, the room impulse response, and the noise signal, respectively. Here, k = 1,..., K represents a frequency index, and t = 1,..., T represents a frame (time) index. It is known that convolution of signals in the time domain can be approximated by convolution of narrowband signals at each frequency of each signal under specific conditions. If the short-time Fourier transform analysis conditions are selected such that this approximation holds, y _k [t] can be expressed as in equation (2).

In the above, a time-invariant system is considered, but since the sound source may actually move, the time-invariant assumption of the indoor transmission system does not necessarily hold. Here, assuming a time-varying indoor transmission system is equivalent to allowing the indoor impulse response h _k [τ] to change at each time t in the equation (2). That is, it is expressed as the following equation (3).

The speech enhancement problem under non-stationary noise and time-varying reverberation environment is based on the following equation (3): observed time frequency component Y = {y _k [t]} _{1 ≦ k ≦ K, 1 ≦ t ≦ T} to S = {s _k [t]} Formulated as a problem of estimating _{1 ≦ k ≦ K and 1 ≦ t ≦ T.} In equation (3), the indoor impulse response is allowed to change with time, so the number of unknown parameters is overwhelmingly larger than in equation (2), and s, h, and n give the same right side. There are countless combinations. Therefore, in the following, we will explain what assumptions should be made in order to obtain a correct solution from the myriad combinations.

<"Half-time degeneration" of indoor transmission system>

  One of the main factors that cause the room impulse response to change with time is the movement of the speaker. This is because as the speaker moves, the distances of all acoustic paths from the speaker to the microphone (including the arrival path of the reflected sound from the wall) change all at once.

  Here, focusing only on a single acoustic path, the impulse response of the transmission system is a delta function, but if there is a change in the distance of the path, the phase component of the Fourier transform of the impulse response is the distance of the acoustic path. The change in arrival time accompanying the change and the amplitude component reflect the change in intensity accompanying the change in the distance of the acoustic path. Since the intensity of sound is inversely proportional to the propagation distance, if the propagation distance is large to some extent, the amplitude component hardly changes even if the propagation distance changes.

  On the other hand, since the phase component changes by an amount proportional to the arrival time and the frequency, even if the change of the arrival time is slight, it can change remarkably especially in the high frequency range.

  Therefore, it is expected that the short-time Fourier transform of the impulse response of the transmission system of a single acoustic path has the property that the amplitude does not easily change and the phase easily changes as the speaker moves.

  Since the actual indoor impulse response is a superposition of the impulse responses of the transmission systems of all acoustic paths, if each impulse response tends to be as described above, the superposition tends to be roughly the same. It is thought that.

  FIG. 1 compares the amplitude component and the phase component of the short-time Fourier transform of the room impulse response measured by changing the position of the microphone by several centimeters instead of the speaker.

  The upper part of FIG. 1 shows the amplitude component (A) and phase component (B) of the short-time Fourier transform of the room impulse response, and the lower part of FIG. 1 shows the measurement by changing the position of the microphone by several centimeters. The amplitude component (C) and phase component (D) of the short-time Fourier transform of the indoor impulse response are shown. The room impulse response data was measured in a reverberation variable room (panel) with a reverberation time of 0.3 [s] recorded in the microphone array database of the RWCP real-world voice and sound database.

  From FIG. 1, it can be seen that the amplitude component does not change so much, while the phase component changes significantly.

  From the above, the inventors can handle minor changes in the environment, such as speaker movement, using a special system that assumes that the amplitude component of the short-time Fourier transform of the room impulse response is time-invariant and the phase component is time-variant. I thought that. Hereinafter, such a system is referred to as a “half-time varying system”.

<Observation signal generation model>

  Based on the above two assumptions, an observation signal generation model is formulated. When the assumption of half-time variation is introduced into the expression (3), the short-time Fourier transform of the observation signal is expressed as the expression (4).

Where φ _{k, t} [τ] and H _k [t] are the phase component and amplitude component of the short-time Fourier transform of the room impulse response, and H _k [t] = | h _{k, t} [τ] | . Here, φ _{k, t} [τ] is a random variable that follows a uniform distribution on the interval [0; 2π) independently for each k, n, and m. Further, the original speech signal and noise are assumed to be random variables according to the complex normal distributions (5) and (6), each having an average of 0.

Here, Φ ^(s) _k [t] and Φ ⁽ⁿ⁾ _k [t] are the power spectral density at the time t and the kth frequency bin of the original speech signal and the noise signal, respectively. Here, N _C (z; 0, λ) represents a probability density function of a complex normal distribution, and is given by equation (7).

Although the proof is omitted, the short-time Fourier transform y _k [t] of the observed signal is independent for each k and t from the assumption of the Gaussian nature of the original speech signal and noise and the reproducibility of the normal distribution. It is shown to follow the complex normal distribution shown in.

Next, a non-negative matrix product representation is introduced for the power spectral density sequence Φ ^(s) _k [t] of the original speech signal. Assume that the number of base spectra is M, and the m-th base spectrum is B ^(s) _{m, k} . Also, ^assuming that the gain of the base spectrum B ^(s) _{m, k} at time t is G ^(s) _m [t], the power spectrum density sequence of the original speech signal is expressed by equations (9) and (10).

  Note that the base spectrum and its gain are non-negative. Moreover, in order to remove the arbitraryness of the scale, the condition shown in the equation (11) is assumed to be satisfied.

The speech base spectrum B ^(s) _{m, k} may be estimated blindly as an unknown variable from the observed signal as an unknown variable, but it is assumed that it is learned in advance from a certain amount of clean speech samples. In that sense, the method according to the present embodiment (hereinafter referred to as the proposed method) is a semi-blind speech enhancement method. Prior learning of the basis can be performed by applying NMF to the spectrogram of clean speech data.

<Parameter estimation algorithm>

The probability density function of the observation time frequency component Y = {y _k [t]} _{1 ≦ k ≦ K and 1 ≦ t ≦ T established above} corresponds to the likelihood function of the unknown parameter. From equation (8), the log likelihood of the unknown parameter with the observation time frequency component Y given is the observation spectrogram | y _k [t] | ² and the equation (12) if the constant term / sign is ignored. It becomes equal to the Itakura Saito distance (13) expression showing the distance with the power spectrum density series model shown.

Therefore, the proposed method results in an optimal fitting problem between the observed spectrogram | y _k [t] | ² and the power spectral density sequence model Φ ^x _k [t]. Here, the power spectrum density sequence Φ ⁽ⁿ⁾ _k [t] of the noise signal is also modeled as in equation (14).

The Itakura Saito distance I (H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ) corresponds to a distance obtained when β is set to 0 in β-divergence representing a distance scale.

The Itakura Saito distance I (H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ) represented by the equation (13) is difficult to directly minimize, so the Itakura Saito distance I (H, G ^{( s)} , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ) Set an upper limit function that touches at one point without falling below. Although the derivation is omitted, the upper limit function shown in the equation (15) is the upper limit function of the Itakura Saito distance I (H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ).

That is, instead of the Itakura Saito distance I (H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ), H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ and auxiliary variables ζ, ξ, η are updated alternately to reduce the objective function I (H, G ^(s) , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ) Can do. At this time, the parameters of H, G ^(s) , B ⁽ⁿ⁾ , and G ⁽ⁿ⁾ are updated while maintaining non-negative values. Note that Φ ^(x) _k [t] in equation (15) is expressed by equation (16).

The update rule of the upper bound function D ₀ is represented by (17) to (23) below.

<System configuration>

  Next, by analyzing an observation signal in which noise and reverberation components are superimposed on the original speech signal, and estimating the original speech signal included in the observation signal, the case where the present invention is applied as an example, An embodiment of the present invention will be described.

  As shown in FIG. 2, the acoustic signal analysis apparatus according to the embodiment of the present invention is a computer that includes a CPU, a RAM, and a ROM that stores a program for executing an acoustic signal analysis processing routine to be described later. It is configured and functionally configured as follows.

  The acoustic signal analysis device 100 includes an input unit 10, a calculation unit 20, a storage unit 30, and an output unit 40.

  The input unit 10 inputs time series data of an observation signal in which noise and reverberation components are superimposed on the original voice signal. The storage unit 30 stores time-series data of observation signals input by the input unit 10. In addition, the storage unit 30 stores the result of each process to be described later, and stores the initial value of each parameter used in this process routine.

  The calculation unit 20 includes a time-frequency analysis unit 21, an initial setting unit 22, an auxiliary variable update unit 23, a parameter update unit 24, an end determination unit 25, and a signal conversion unit 26.

The time frequency analysis unit 21 receives, for example, an observed observation signal y [t] as a time series signal of a microphone, and receives an observation time frequency component Y = {y _k [t]} (frequency) of the observation signal y [t]. k = 1,..., K, time t = 1,. The calculated observation time frequency component Y is stored in the storage unit 30. More specifically, the time-frequency analysis unit 21 performs time-frequency analysis using, for example, short-time Fourier transform (STFT) using time-series data of observation signals observed with a microphone as input. The observation time frequency component Y is calculated by

The initial setting unit 22 uses initial values of each parameter G ^(s) _m [t], H _k [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [t] and B used in processing to be described later. ^(s) Set the _{m and k} values. The initial values of parameters other than B ^(s) _{m, k} may be set to appropriate values using, for example, random numbers. In this case, the initial values of the parameters G ^(s) _m [t], H _k [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [t] are set to be non-negative values.

A value obtained by learning in advance from a certain amount of clean speech samples is set as the value of B ^(s) _{m, k} . It is assumed that the value of B ^(s) _{m, k} has been learned in advance so as to satisfy the above equation (11) in order to eliminate scale arbitraryness.

The auxiliary variable updating unit 23 stores B ^(s) _{m, k} , G ^(s) _m [t], H stored in the storage unit 30 for each of all combinations of (k, m, t, τ). _{Based on k} [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [t], the auxiliary variable ζk _{, m, t, τ is set} so as to reduce the upper limit function D ₀ according to the above equation (21). Is updated and stored in the storage unit 30.

The auxiliary variable updating unit 23 also stores B ^(s) _{m, k} , G ^(s) _m [t], H stored in the storage unit 30 for each of all combinations of (k, q, t). _{Based on k} [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [t], the auxiliary variable ξk _{, q, t} is updated so as to reduce the upper limit function D ₀ according to the above equation (22). And stored in the storage unit 30.

Further, the auxiliary variable update unit 23 stores B ^(s) _{m, k} , G ^(s) _m [t], H _k [τ], B stored in the storage unit 30 for each time t and each frequency k. ^{(n) Based on} _{q, k} and G ⁽ⁿ⁾ _q [t], the auxiliary variable η _k [t] is updated so as to reduce the upper limit function D ₀ according to the above equations (16) and (23). And stored in the storage unit 30.

The parameter update unit 24 stores, for each combination of (m, t), y _k [t], B ^(s) _{m, k} , H _k [τ], ζ _{k, Based on m, t, τ} and η _k [t], the gain coefficient G ^(s) _m [of the base spectrum B ^(s) _{m, k} is set so as to reduce the upper limit function D ₀ according to the above equation (17). t] is updated and stored in the storage unit 30.

The parameter update unit 24 also stores y _k [t], G ⁽ⁿ⁾ _q [t], ξ _{k, q, t} stored in the storage unit 30 for each of all combinations of (q, k). , Η _k [t], the element B ⁽ⁿ ) of the basis matrix of the power spectral density Φ ⁽ⁿ⁾ _k [t] of the noise signal n so as to reduce the upper limit function D ₀ according to the above equation (18) ⁾ _{q and k} are updated and stored in the storage unit 30.

In addition, the parameter update unit 24 uses y _k [t], B ⁽ⁿ⁾ _{q, k} , ξ _{k, q, t} , stored in the storage unit 30 for each of all combinations of (q, t). Based on η _k [t], the gain G ⁽ⁿ⁾ _q [t] of the base spectrum B ⁽ⁿ⁾ _{q, k} is updated and stored so as to reduce the upper limit function D ₀ according to the above equation (19). Store in the unit 30.

Further, the parameter update unit 24 stores y _k [t], B ^(s) _{m, k} , G ^(s) _m [t] stored in the storage unit 30 for each of all combinations of (k, τ). ], Ζ _{k, m, t, τ} and η _k [t], the power H _k [τ] of the indoor impulse response is updated so as to reduce the upper limit function D ₀ according to the above equation (20). And stored in the storage unit 30.

The end determination unit 25 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, the processes of the auxiliary variable update unit 23 and the parameter update unit 24 are repeated. When it is determined that the end condition is satisfied, the end determination unit 25 proceeds to processing by the signal conversion unit 26. Based on B ^(s) _{m, k} and G ^(s) _m [t] stored in the storage unit 30, the signal conversion unit 26 estimates an original speech signal (hereinafter, estimated) by removing a noise signal. The output unit 40 outputs an estimated original voice signal.

  As an end condition, the difference between the value of the upper limit function (15) with the number of repetitions L-1 and the value of the upper limit function (15) with the number of repetitions L is less than a predetermined threshold. What has become smaller can be used. Alternatively, the termination condition may be that the number of repetitions has reached a predetermined upper limit number.

<Operation of acoustic signal analyzer>

  Next, the operation of the acoustic signal analysis apparatus 100 according to the present embodiment will be described. First, as a signal to be analyzed, time-series data of an observation signal in which noise and reverberation components are superimposed on an original speech signal is input to the acoustic signal analysis device 100 and stored in the storage unit 30. Then, in the acoustic signal analysis apparatus 100, an acoustic signal analysis processing routine shown in FIG. 3 is executed.

First, in step S101, the observation signal y [t] is read from the storage unit 30, and the observation signal y [t] is subjected to time-frequency analysis using short-time Fourier transform for the observation signal y [t]. Observation time frequency component Y = {y _k [t]} (k = 1,..., K, t = 1,..., T) and the obtained observation time frequency component Y is stored. Store in unit 30.

In step S102, initial values of G ^(s) _m [t], H _k [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [t] are set using random numbers. And each element of the base matrix consisting of M base spectra m of each frequency k obtained by learning in advance from a fixed amount of clean speech samples is set as B ^(s) _{m, k} And stored in the storage unit 30.

In step S103, B ^(s) _{m, k} , G ^(s) _m [t], H _k [τ], B ⁽ⁿ⁾ _{q, k} , G ⁽ⁿ⁾ _q [ t], or B ^(s) _{m, k} set in step S102 and G ^(s) _m [t], H _k [τ], B ⁽ updated in steps S104, S105, S106, and S107 described later. ^{n) Based on} _{q, k} , G ⁽ⁿ⁾ _q [t], the auxiliary variable ζ _{k, m, t, τ} is calculated for each combination of (k, m, t, τ) according to the above equation (21). In addition, the auxiliary variables ξ _{k, q, t} are calculated for each combination (k, q, t) according to the above equation (22), and further, according to the above equations (16), (23) _k [t] is calculated for each time t and each frequency k and stored in the storage unit 30.

In step S104, the observation time frequency component Y of the observation signal y [t] generated in step S101, B ^(s) _{m, k} set in step S102, and H _k set in step S102. [τ] or H _k [τ] updated in step S107, which will be described later, and the auxiliary variables ζ _{k, m, t, τ} , η _k [t] updated in step S103, (17 ) _, A non-negative gain coefficient G ^(s) _m [t] of the base spectrum B ^(s) _{m, k} is calculated for each combination of (m, t) and stored in the storage unit 30.

In step S105, the observation time frequency component Y of the observation signal y [t] generated in step S101, G ⁽ⁿ⁾ _q [t] set in step S102, or updated in step S106 described later. Based on G ⁽ⁿ⁾ _q [t] and the auxiliary variables ξ _{k, q, t} and η _k [t] updated in step S103, the power spectral density Φ of the noise signal n is obtained according to the above equation (18). ⁽ⁿ⁾ Element B ⁽ⁿ⁾ _{q, k} of the basis matrix of _k [t] is calculated for each (q, k) combination and stored in the storage unit 30.

In step S106, the observation time-frequency component Y of the generated observation signal y [t] in step S101, updated in step S102 is set in the B ⁽ⁿ⁾ _{q, k} or step S105, B ^{( n) Based on} _{q, k} and the auxiliary variables ξ _{k, q, t} and η _k [t] updated in step S103, the gain of the base spectrum B ⁽ⁿ⁾ _{q, k} according to the above equation (19) G ⁽ⁿ⁾ _q [t] is calculated for each (q, t) combination and stored in the storage unit 30.

In step S107, the observation time frequency component Y of the observation signal y [t] generated in step S101, B ^(s) _{m, k} set in step S102, and the step S102 are set. _{^{G (s) m [t]}} , or the G ^(s) _m [t] updated in step S104, the auxiliary variables updated in step _{S103 ζ k, m, t,} τ, η k [t] Based on the above, the power H _k [τ] of the room impulse response is calculated for each combination (k, τ) according to the above equation (20) and stored in the storage unit 30.

In the next step S108, the observation time frequency component Y of the observation signal y [t] generated in step S101, B ^(s) _{m, k} set in step S102, and updated in step S103. Auxiliary variables ζ _{k, m, t, τ} , ξ _{k, q, t} , η _k [t], G ^(s) _m [t] updated in step S104, and B updated in step S105 ^{Based on (n)} _{q, k} , G ⁽ⁿ⁾ _q [t] updated in step S106, and H _k [τ] updated in step S107, the upper limit function D is determined according to the above equation (15). A value of ₀ is calculated and stored in the storage unit 30. Then, read the value of the upper bound function D ₀ calculated in the previous step S108 from the storage unit 30, the value of the upper bound function D ₀ calculated in this step S108, the value of the upper bound function D ₀ calculated in the previous step S108 Is determined to be smaller than a predetermined threshold stored in the storage unit 30 in advance, and if the difference is equal to or larger than the predetermined threshold, the end condition is not satisfied. Determination is made, the process returns to step S103, and the processes of steps S103 to S107 are repeated. On the other hand, if the difference is less than the predetermined threshold value, it is determined that the end condition is satisfied, and B ^(s) _{m, k} set in step S102 and the final result in step S104 are determined in step S109. The estimated original speech signal is calculated based on the updated G ^(s) _m [t] and the estimated original speech signal is output from the output unit 40, and the acoustic signal analysis processing routine is terminated.

<Results>

  Next, for the purpose of showing the usefulness of the technique according to the present embodiment, the PHS ringtone, background noise, the PHS ringtone, and the background are added to the moving voice signal in the reverberation room (reverberation time 1.3 seconds). A description will be given of the results of an evaluation experiment using the proposed method when a non-stationary noise such as a noise combined sound, a particle falling sound, a BGM sound, or another person's voice is superimposed as an observation signal.

  FIG. 4 is a diagram showing an evaluation experiment result when the proposed method is used. FIG. 4A shows the experimental results based on Mel-Frequency Cepstral Coefficients Distortion (MFCCD) as an evaluation standard, and FIG. 4B shows the Bark Spectral Distortion Score (BSDS). FIG. 4C shows the experimental results using the signal-to-interference ratio (SIR) as the evaluation standard. Note that MFCCD and BSDS are values indicating that the sound clarity is improved as the values are reduced, while SIR is a value indicating that the sound clarity is improved as the values are increased.

  As can be seen from FIG. 4 above, except for the evaluation result of the MFCCD when the voice of the BGM or another person is superimposed on the mobile voice signal, the estimated original voice signal of the proposed method is compared with the clarity of the voice included in the observed signal. The result that the clarity was improved was obtained. In particular, in the evaluation result regarding SIR when the PHS ringtone is superimposed on the mobile voice signal, an improvement of 4 times or more was observed.

As described above, according to the acoustic signal analysis device according to the embodiment of the present invention, each element G ^(s) _m of the coefficient matrix is set so that the value of the upper limit function expressed by the above equation (15) becomes small. [t], each element B ⁽ⁿ⁾ _{q, k of} the basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and the power H _k [ By repeatedly updating τ], auxiliary variable ζ _{k, m, t, τ} , auxiliary variable ξ _{k, q, t} and auxiliary variable η _k [t], noise and reverberation components are superimposed on the audio signal. The speech signal can be accurately estimated from the observed signal and the clarity of the speech signal can be improved.

  Thus, in the method proposed in the present invention, with respect to the movement of the sound source, etc., an observation signal generation model in which the amplitude component of the short-time Fourier transform of the room impulse response is time-invariant and the phase component is time-variable is established and observed. Estimate the original speech signal included in the signal with high accuracy

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above-described acoustic signal analysis apparatus has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

Further, in the method according to the present embodiment, the power spectral density series model Φ ^x _k [t] and the observation time-frequency component y _k [t] Itakura Saito as a measure of the distance between the distance ^{I (H, G (s)} , B ⁽ⁿ⁾ , G ⁽ⁿ⁾ ) are used, but the present invention is not limited to this, and a scale such as a Generalized Kullback-Leibler distance or Frobenius norm, which is a kind of β-divergence, may be used.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 21 Time frequency analysis part 22 Initial setting part 23 Auxiliary variable update part 24 Parameter update part 25 End determination part 26 Signal conversion part 30 Storage part 40 Output part 100 Acoustic signal analysis apparatus

Claims (5)

A time-frequency analysis unit that outputs time-series data of an acoustic signal and outputs an observation time-frequency component y _k [t] of each frequency k at each time t;
The power spectrum density Φ ^(s) _k [t] at each time t and each frequency k of the original speech signal s included in the acoustic signal is represented by each non-negative element B ^(s) _m representing M base spectra _{m. , k} and the coefficient matrix with each non-negative element as a gain matrix G ^(s) _m [t] of the base spectrum B ^(s) _{m, k} at each time t Each element G ^(s) _m [t] of the coefficient matrix, power H _k [τ] of each frequency k of the indoor impulse response at each time τ, each time t and each frequency of the noise signal n included in the acoustic signal The power spectral density Φ ⁽ⁿ⁾ _k [t] of _k is represented by a base matrix composed of non-negative elements B ⁽ⁿ⁾ _{q, k} representing Q base spectra q and a base spectrum B ^{(n at} each time t. ⁾ Each element B ⁽ⁿ⁾ _{q, k of} the base matrix when the gain G ⁽ⁿ⁾ _q [t] of _{q, k} is expressed as a product with a coefficient matrix having each element as a non-negative value The key points of Sets the initial value of each of the _{^{G (n) q [t]}} , the element B ^(s) _m the base _matrix, the _k, the parameter initial value setting unit for setting a pre-determined value,
The element B ^(s) _{m, k of the} basis matrix, the element G ^(s) _m [t] of the coefficient matrix, the power H _k [τ], and the power in all combinations of (τ, m) Each of the power spectral density series model Φ ^x _k [t] calculated on the basis of the spectral density Φ ⁽ⁿ⁾ _k [t] and the observation time frequency component y _k [t] are reduced in distance. The observation time frequency component y _k [t] at time t and each frequency k, each element B ^(s) _{m, k of the} basis matrix, each element G ^(s) _m [t] of the coefficient matrix, Each element B ⁽ⁿ⁾ _{q, k of the} base matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and power H _k [τ] of each frequency k of the indoor impulse response at each time τ , Each element G ^(s) _m [t] of the coefficient matrix and each element B ^{(n of the} basis matrix ⁾ based on the power spectral density series model Φ ^x _k [t] at each time t and each frequency k ⁾ _{q, k} and each element G ^{(n) of the} coefficient matrix a parameter updating unit that updates _q [t] and the power H _k [τ] of each frequency k of the indoor impulse response at each time τ;
An end determination unit that repeatedly performs update by the parameter update unit until a predetermined end condition is satisfied;
An acoustic signal analyzing apparatus including: The acoustic signal analysis apparatus according to claim 1, wherein β-divergence is used as a measure of the distance between the power spectral density series model Φ ^x _k [t] and the observation time frequency component y _k [t]. An auxiliary variable update unit,
A function representing the distance between the power spectral density series model Φ ^x _k [t] and the observation time frequency component y _k [t] is expressed as the observation time frequency component y _k [t] at each time t and each frequency k. Each element B ^(s) _{m, k of the} basis matrix, each element G ^(s) _m [t] of the coefficient matrix, each element B ⁽ⁿ⁾ _{q, k of the} basis matrix _, and the coefficient Each element G ⁽ⁿ⁾ _q [t] of the matrix, the power H _k [τ] of each frequency k of the indoor impulse response at each time τ, and the power spectrum density series model Φ ^x _{k at} each time t and each frequency _k [t] and auxiliary variables ζ _{k, m, t, τ} for all combinations of (k, m, t, τ) and auxiliary variables ξ _k, for all combinations of (k, q, t) _{q, t} and auxiliary variables η _k [t] for each time t and each frequency k, and the power spectral density series model Φ ^x _k [t] and the observation time frequency component y _k β-diver with [t] And auxiliary function, which is the upper limit function of ence,
The auxiliary variable updating unit reduces each auxiliary function to each element B ^(s) _{m, k of the} basis matrix, each element G ^(s) _m [t] of the coefficient matrix, and the basis matrix Element B ⁽ⁿ⁾ _{q, k} , each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, power H _k [τ] of each frequency k of the room impulse response at each time τ, and each time Based on t and the power spectral density series model Φ ^x _k [t] of each frequency k, auxiliary variables ζ _{k, m, t, τ} for all combinations of (k, m, t, τ), ( update auxiliary variables ξ _{k, q, t} for all combinations of k, q, t) and auxiliary variables η _k [t] for each time t and each frequency k,
The parameter update unit reduces the auxiliary function so that the observation time frequency component y _k [t] at each time t and each frequency k, and each element B ^(s) _{m, k of the} basis matrix, Each element G ^(s) _m [t] of the coefficient matrix, each element B ⁽ⁿ⁾ _{q, k of the} basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and each time τ And the auxiliary variables ζ _{k, m, t, τ} for all combinations of the power H _k [τ] of each frequency k of the room impulse response of (k, m, t, τ) _, and (k, q, t ) based on the auxiliary variables xi] _{k, q} for all _{combinations,} and _t, an auxiliary variable eta _k [t] for each time t and each frequency k of the elements G ^(s of the coefficient ^matrix) _m [ t], each element B ⁽ⁿ⁾ _{q, k of the} basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and power H _k of each frequency k of the room impulse response at each time τ The acoustic signal analyzing apparatus according to claim 2, wherein [τ] is updated. The time-frequency analysis unit inputs time series data of the acoustic signal, and outputs an observation time frequency component y _k [t] of each frequency k at each time t,
The parameter initial value setting unit converts the power spectrum density Φ ^(s) _k [t] at each time t and each frequency k of the original audio signal s included in the acoustic signal to a non-negative value representing M base spectra m. A basis matrix composed of each element B ^(s) _{m, k} , a coefficient matrix having a gain G ^(s) _m [t] of the base spectrum B ^(s) _{m, k} at each time t as a non-negative element, Each coefficient G ^(s) _m [t], power H _k [τ] of each frequency k of the room impulse response at each time τ, and noise signal n included in the acoustic signal Power spectrum density Φ ⁽ⁿ⁾ _k [t] at each time t and each frequency k, a base matrix composed of non-negative elements B ⁽ⁿ⁾ _{q, k} representing Q base spectra q, and each time each of the basis matrix when expressed basal spectrum B ⁽ⁿ⁾ _q in _t, the gain of the _k G a ⁽ⁿ⁾ _q [t] as the product of the coefficient matrix with each element is non-negative Containing B ⁽ⁿ⁾ _{q, k,} sets the respective initial values of the elements G ⁽ⁿ⁾ _q [t] of the coefficient matrix, the elements B ^(s) _m the base _matrix, the _k, determined in advance Set the value
By the parameter updating unit, the element B ^(s) _{m, k of the} basis matrix, the element G ^(s) _m [t] of the coefficient matrix, and the power H _k [τ] in all combinations of (τ, m). ] And the power spectral density series model Φ ^x _k [t] calculated based on the power spectral density Φ ⁽ⁿ⁾ _k [t] and the observation time frequency component y _k [t] are small The observation time frequency component y _k [t] at each time t and each frequency k, each element B ^(s) _{m, k of the} basis matrix _, and each element G ^(s) _{m of the} coefficient matrix [t], each element B ⁽ⁿ⁾ _{q, k of the} basis matrix, each element G ⁽ⁿ⁾ _q [t] of the coefficient matrix, and power H of each frequency k of the room impulse response at each time τ _k [τ] and the power spectral density series model Φ ^x _k [t] at each time t and each frequency k, each element G ^(s) _m [t] of the coefficient matrix and the basis matrix each element B ⁽ⁿ⁾ _q, and _k Wherein each element G of the coefficient matrix ⁽ⁿ⁾ _q [t], and updates the power H _k [tau] of each frequency k of the room impulse response for each time tau,
An acoustic signal analysis method in which updating by the parameter updating unit is repeatedly performed by a termination determination unit until a predetermined termination condition is satisfied.   The program for functioning a computer as each part of the acoustic signal analyzer of any one of Claims 1-3.