JP2014048399A - Sound signal analyzing device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound signal analyzing device, method and program capable of precisely separating a time-series data of sound signals output from a plural microphones into sound source signals of each sound source even when the number of the sound sources is unclear.SOLUTION: A parameter update section 23 defines a sound source which gets active at each time t with respect to each frequency ω; models a prior distribution of the probability that each sound source k gets active in a stick-breaking construction; models a prior distribution of an indicator representing an index in an arrival direction of each sound source k using a discrete probability distribution; models a prior distribution of the probability when each sound source k is the arrival direction of each index using a Dirichlet distribution; and repeatedly updates, on the basis of a variation estimation, parameters of a variable function approximating a posterior distribution of the parameter of generation model of the sound source signal. A time frequency element estimation section 26 estimates the time frequency element of the sound source signals of each sound source k by using the final-updated parameter.

Description

  The present invention relates to an acoustic signal analyzing apparatus, method, and program, and more particularly, to an acoustic signal analyzing apparatus, method, and program for separating time-series data of acoustic signals output from a plurality of microphones into signals of respective sound sources. .

  A technique that separates and extracts individual sound source components from the microphone input signal when both the sound source components and the transfer characteristics from the sound source to the microphone are unknown is called Blind Source Separation (BSS). In BSS, it is necessary to estimate the sound source signal and its mixing process only from the observed signal. Therefore, it is usually formulated as an optimization problem in which both unknown variables are estimated based on the criteria established by making some assumptions about the sound source. Is done. For example, when the number of observation signals is equal to or greater than the number of sound sources, a method called independent component analysis in which the separation filter is maximum likelihood estimated on the assumption that the sound source signal components follow a Gaussian distribution is well known. However, in order to realize a BSS system that does not assume the number of sound sources, it is necessary to assume an indeterminate problem setting in which the number of sound sources is larger than the number of microphones, and independent component analysis cannot be applied as it is. Under underdetermined conditions, even if the mixing process is known, the solution cannot be determined uniquely, so a stronger assumption than the independence is required for the sound source.

  Underdetermined BSS is a difficult defect setting problem, but when the number of sound sources is known, in the time-frequency domain using the property called sparseness that the voice energy becomes dominant only in the time-frequency domain. In recent years, the BSS approach has attracted attention as one of the effective approaches (Non-Patent Document 1). The sparseness of speech is a property in which the time frequency component of the speech signal is almost zero in many regions. For this reason, even when a plurality of voices are spoken simultaneously, it is often assumed that the time frequency components of the voices hardly overlap each other at each time frequency. Based on this assumption, how to design a time-frequency mask that allows only the time-frequency component of the target speech signal to pass is the focus of the problem in this approach. In addition, this approach needs to solve a problem called permutation matching that combines components separated for each frequency for each sound source. In the prior art, permutation matching is a post-processing of signal separation for each frequency. It was often done as.

H. Sawada, S. Araki, and S. Makino, “Underdetermined convolutive blind source separation via frequency bin-wise clustering and permutation alignment,” IEEE Trans. ASLP, vol. 19, no. 3, pp. 516-527, 2010 .

  BSS for audio signals is expected to have many applications such as hands-free video conferencing system and automatic system for creating conference contents. For example, in a meeting situation, it is difficult to assume the number of all sound sources in advance in an actual environment so that the number of participants may change midway or a door opening / closing sound may suddenly sound. Many conventional BSS algorithms operate assuming the number of sound sources, and if the assumed number of sound sources is different from the actual number of sound sources, high performance may not be achieved.

  Therefore, it is desirable to realize a BSS system that operates while inferring the number of sound sources autonomously without assuming the number of sound sources. When the number of sound sources is known, the BSS approach in the time frequency domain using the property called sparsity that the energy of the voice is dominant only in the time frequency domain is effective, but the number of sound sources is unknown In this case, how to solve the problem of BSS becomes a problem.

  The observed signal at each microphone is usually represented by convolutional mixing including the time delay of the sound source signal, but it is a time-frequency decomposition (short-time Fourier transform) with a sufficiently long time window for the impulse response length from the sound source to the microphone. , Wavelet transform, etc.), convolutional mixing can be approximated by instantaneous mixing. BSS based on this observation model is called frequency domain BSS, and features such as the ability to implement an algorithm with a small amount of computation and the above-mentioned assumption of sparsity of speech compared to BSS based on the time domain convolution mixed model On the other hand, there is a need to deal with a problem called permutation matching for collecting components separated for each frequency for each sound source. In the conventional approach, permutation matching is often performed as a subsequent process of signal separation for each frequency. However, the relationship between the frequencies of the sound source and the spectrum structure, which are clues in permutation matching, are determined by the signal separation at each frequency. It should be able to contribute to accuracy improvement. Therefore, how to solve the permutation matching problem and the sound source separation simultaneously becomes a problem.

  The present invention has been made in view of the above circumstances, and accurately separates sound source signals for each sound source from time-series data of acoustic signals output from a plurality of microphones even if the number of sound sources is unknown. It is an object to provide an acoustic signal analysis apparatus, method, and program capable of performing

In order to achieve the above object, the acoustic signal analyzing apparatus according to the present invention receives time series data of acoustic signals output from M (M is an integer of 2 or more) microphones m as input, and an observation time frequency component y. Time frequency analysis means for outputting a three-dimensional array y ^ having elements _{m, ω, t} (m is a microphone, t is a time, and ω is an index of frequency), and D (D Posterior of each vector a _{k, ω} ^ included in the three-dimensional array a ^ having elements of the transmission frequency characteristics a _{m, k, ω} of the sound source signal from each of the sound sources k of 1) to each microphone m Parameters of distribution (average vector m _{k, ω} ^ of complex normal distribution, covariance matrix Γ _{k, ω} ), two-dimensional element having time frequency components s _{ω, t} of active sound source at each time t for each frequency ω each element in the array s ^ s _ω, of the posterior distribution of _t Parameters (mean mu _omega of complex normal _{distribution, t,} variance σ ² _{ω, t),} each frequency indicator indicates the index of the sound source which becomes active at each time t with respect to omega z _omega, two-dimensional array having a _t the element z ^ Posterior distribution parameters of each element z _{ω, t} of each element (discrete probability distribution parameters φ _{k, ω, t} ), and each element v _k of the vector v ^ having the probability v _k that each sound source k is active Posterior distribution parameters (beta distribution parameters γ _{k, 0} , γ _{k, 1} ) and posterior distribution of each element x _k of vector x ^ having an indicator x _k indicating the arrival direction index i of each sound source k. Parameter (each parameter ψ _{k, i of the} discrete probability distribution), and the posterior distribution of each vector ρ _k ^ of the two-dimensional array ρ ^ having the probability ρ _{k, i} of each sound source k being the arrival direction of each index i Parameters (each parameter of the directory distribution _k, an initial setting means for setting each initial value of _i), the three-dimensional array y ^ wherein when given a three-dimensional array a ^, the two-dimensional array s ^, the two-dimensional array z ^, The vector v ^, the vector x ^, and the posterior distribution p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) and the variable function q (a ^) of the two-dimensional array ρ ^ , S ^, z ^, v ^, x ^, ρ ^), a function representing a divergence representing a difference between the objective function and the objective function is (k M), the mean vector m _{k, ω} ^ and the variance Γ _{k, ω} in each of all combinations _, and the mean μ _{ω, t} and the variance σ ^{2 in} each of all combinations of (ω, t). _The parameters φ _{k, ω, t} in each of all combinations of _{ω, t} and (k, ω, t) and the parameters in all k Parameter updating means for updating the data γ _{k, 0} , γ _{k, 1} , the parameter ψ _{k, i,} and the parameter ζ _{k, i} in each of all combinations of (k, i), For each of all combinations of (k, ω, t) and end determination means for repeatedly updating by the parameter update means until an end condition is satisfied, the parameter φ _{k, ω, t} and the average μ _{ω, t} And a sound source signal estimating means for estimating a time frequency component s _{k, ω, t} of the sound source signal of the sound source k.

In the acoustic signal analysis method according to the present invention, the time-frequency analysis means receives time-series data of acoustic signals output from M (M is an integer of 2 or more) microphones m as input, and the observation time-frequency component ym _{, A} three-dimensional array y ^ having _{ω, t} (m is a microphone, t is a time, and ω is an index of frequency) as elements is output, and D is set (D is 1 for each frequency ω) by the initial setting means. (Integer above) from each of the sound sources k to each microphone m, the posterior distribution of each vector a _{k, ω} ^ included in the three-dimensional array a ^ having the transmission frequency characteristics a _{m, k, ω} of the sound source signal as elements. Parameters (average vector m _{k, ω} ^, covariance matrix Γ _{k, ω} ) of complex normal distribution, two-dimensional array s having elements of time frequency components s _{ω, t} of active sound sources at each time t for each frequency ω ^ each of the elements included in the s _ω, the posterior distribution of _t Parameters (mean of complex normal distribution mu _{omega, t,} variance σ ² _{ω, t),} each frequency indicator indicates the index of the sound source which becomes active at each time t with respect to omega z _omega, two-dimensional array having a _t the element z ^ Posterior distribution parameters of each element z _{ω, t} of each element (discrete probability distribution parameters φ _{k, ω, t} ), and each element v _k of the vector v ^ having the probability v _k that each sound source k is active Posterior distribution parameters (beta distribution parameters γ _{k, 0} , γ _{k, 1} ) and posterior distribution of each element x _k of vector x ^ having an indicator x _k indicating the arrival direction index i of each sound source k. Parameter (each parameter ψ _{k, i of the} discrete probability distribution), and the posterior distribution of each vector ρ _k ^ of the two-dimensional array ρ ^ having the probability ρ _{k, i} of each sound source k being the arrival direction of each index i Parameters (each parameter of the directory distribution Set the initial value of each of the zeta _{k, i),} the parameter by the update means, the three-dimensional array y ^ wherein when given a three-dimensional array a ^, the two-dimensional array s ^, the two-dimensional array z ^, The vector v ^, the vector x ^, and the posterior distribution p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) and the variable function q ( a ^, s ^, z ^, v ^, x ^, ρ ^) as a function representing the divergence representing the difference between the objective function and minimizing the objective function based on the variational inference method. The mean vector m _{k, ω} ^ and the variance Γ _{k, ω} in each of all combinations of (k, m) and the mean μ _{ω, t} and the variance in each of all combinations of (ω, t). σ ² _{ω, t} and the parameter φ _{k, ω, t} in each of all combinations of (k, ω, t) and all k The parameters γ _{k, 0} , γ _{k, 1} , the parameter ψ _k, and the parameters ζ _{k, i} in each of all combinations of (k, i) are updated and predetermined by the end determination means Until the end condition is satisfied, the updating by the parameter updating unit is repeatedly performed, and the parameter φ _{k, ω, t} and the average μ _ω, Based on _t , the time frequency component s _{k, ω, t} of the sound source signal of the sound source k is estimated.

  The program according to the present invention is a program for causing a computer to function as each means of the acoustic signal analyzing apparatus.

  As described above, according to the acoustic signal analysis apparatus, method, and program of the present invention, one sound source is activated at each time t for each frequency ω, and the probability that each sound source k is activated is represented by the beta distribution. The posterior of the parameters of the generation model of the sound source signal that generates the indicator indicating the arrival direction index of each sound source k with a discrete probability distribution and generates the probability that each sound source k becomes the arrival direction of each index with a directory distribution By updating each parameter based on the variational inference method so as to minimize the objective function based on the distribution and estimating the posterior distribution of the time frequency component of the sound source signal of each sound source k, the number of sound sources is unknown. However, it is possible to accurately separate sound source signals for each sound source from time-series data of acoustic signals output from a plurality of microphones. Obtained.

It is the schematic which shows the structure of the acoustic signal analyzer which concerns on embodiment of this invention. It is a flowchart which shows the content of the acoustic signal analysis process routine in the acoustic signal analyzer which concerns on embodiment of this invention. It is a figure which shows the relationship between SDR and K or K * obtained about the conventional method and the proposed method. It is a graph which shows the phase difference between channels of the indoor impulse response utilized for the synthesis | combination of an observation signal. It is a graph which shows the phase difference between channels computed from the estimated transmission frequency characteristic.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An object of the present invention is to solve at once the problems of (1) inference of the number of sound sources, (2) underdetermined frequency domain BSS assuming the sparseness of speech, and (3) permutation matching. The method proposed in the present invention is based on the posterior probability of which sound source seems to be active in each time-frequency bin under the given observation signal, how much each sound source is likely to be active (how many sound sources) Variational Bayesian method), posterior distribution of time-frequency components of each sound source, posterior distribution of steering vector of each sound source, and posterior probability of each sound source from which direction This is achieved by estimation based on

<Principle of the invention>
First, the principle of the present invention will be described.

<Observation model>
First, the observation model will be described. Considering the case of observing sound source signals coming from K signal sources (K is an integer of 1 or more) with M (M is an integer of 2 or more) microphones, the time frequency of the signal observed with the mth microphone Let y _m (ω, t) be the component, s _k (ω, t) be the time-frequency component of the _kth sound source signal, and y ^ (ω, t) = (y ₁ (ω, t), ..., y _M (ω, t)) ^T ∈C ^M , s ^ (ω, t) = (s ₁ (ω, t), ..., s _K (ω, t)) ^T ∈C ^K However, 1 ≦ ω ≦ Ω and 1 ≦ t ≦ T are indices corresponding to frequency and time, respectively. As described above, in the time-frequency domain, the observed signal y ^ (ω, t) can be approximately expressed in the form of instantaneous mixture of s ^ (ω, t) as in the following equation (1). . Note that “＾” attached to a symbol indicates that the symbol is a matrix, a multidimensional array, or a vector.

Matrix a ^ (ω) = (a _{m, k} (ω)) _{M × K} = (a ₁ (ω),... With elements of the transfer frequency characteristics a _{m, k} (ω) from the signal source k to the microphone m. .., a _K (ω)) ∈C ^{M × K} is called a mixing matrix and is assumed to be time invariant below. n ^ (ω, t) represents a component that cannot be expressed as a time-invariant transfer characteristic, such as background noise coming from many directions or a reverberation component exceeding the frame length. Here, when the sparseness of the speech can be assumed, the sound source index that becomes active (dominant) in each time frequency bin (ω, t) is represented by z _{ω, t} ∈ {1, ..., K}. Then, the above equation (1) can be rewritten as the following equation (2).

In this observation model, since it is assumed that all components other than the z _{ω, t} th sound source are 0, only one variable representing the sound source component is sufficient for each time frequency bin. For this reason, the index k of s _k (ω, t) is omitted in the above equation (2). In other words, s (ω, t) is not a specific sound source component but a variable representing a single sound source component active in each time frequency bin. In order to save space on the page, ω and t will be expressed as subscripts.

<Generation model>
<Observation signal generation process>
Based on the observation model described above, the process for generating the observation signal is described by the generation model.

First, assuming that the noise component n _{ω, t} ^ follows a complex normal distribution with mean 0 ^ and covariance Σ ⁽ⁿ⁾ _ω , a _{1: K, ω} ^ = {a _{1, ω} ,. ., a _{K, ω} }, s _{ω, t} and which sound source is active in each time frequency bin, that is, z _{ω, t} is known, y _{ω, t} Is generated according to the following equation (3).

<Infinite number of sound source mixed models>
In the above, the process of generating an observation signal under _{the condition} that z _{ω, t} is known is assumed, but normally, information regarding which sound source is active in each time frequency bin cannot be observed. Therefore, in this embodiment, the indicator z _{ω, t} indicating the active sound source index is regarded as a latent variable, and the generation process is modeled. First, when the number of sound sources is K, z _{ω, t} is _expressed by the following equation (4) in which any index is selected from a set of sound source indexes {1, ..., K} according to a discrete distribution. Assume that it is generated by a process.

Here, π ^ = (π,..., Π _K ) is a probability value indicating the ease of selecting each index in the discrete distribution, and Σ ^K _{k = 1} π _k = 1. Furthermore, it is assumed that these probability values are generated by the Dirichlet distribution shown in the following equation (5).

  Up to this point, we assumed a finite number of sound sources, but if we take the limit of K → ∞, the above equations (3), (4), and (5) become Dirichlet process mixed models. These can be expressed by the following formulas (6) to (9).

GEM (α ₀ ) is the Dirichlet process (see reference (TS Ferguson, “A Bayesian analysis of some nonparametric problems,” Annals of Statistics, vol. 1, no. 2, pp. 209-230, 1973)) One concrete construction method of π ₁ , π ₂ , ... generated by (referred to as the rod folding process (reference (J. Sethuraman, “A constructive definition of Dirichlet priors,” Statistica Sinica, vol. 4 , pp. 639-650, 1994.))) and is given according to the following equations (10) and (11).

Π ₁ , π ₂ , ... generated by the above process mean, in an average sense, π _k tends to decrease exponentially as _k increases, so the sound source corresponding to large k is more active This means that the probability of becoming low becomes low. Therefore, when the parameter is inferred from the observed signal, there is an effect of trying to explain the observed signal with a mixed model having the minimum number of sound source indexes. The above generation model of y _{ω, t} is called an “infinite number of sound source mixture model”.

<Mixed DOA model>
In order to incorporate the permutation matching function into the above model, the generation process of the above equation (9) is modeled below. Up to this point, the transmission frequency characteristics a _{k, ω} of each sound source have been treated as if they were independent variables for each frequency ω, but if it can be assumed that each sound source is a plane wave coming from a single direction, for example, the number of microphones 2 is expressed explicitly as a function of the direction of arrival (Direction-of-Arrival; DOA) θ as shown in the following equation (12).

However, 0 ≦ θ <2π, h _ω is an angular frequency (rad / s) corresponding to the frequency index ω, D is a microphone interval (m), and C is a sound velocity (m / s). Actually, a _{k, ω} is expected to deviate from the above theoretical formula due to the effects of reverberation and instantaneous mixing approximation in the frequency domain. Therefore, when the arrival direction θ _k is known, a _{k, ω} is

Is generated from a complex normal distribution centered at. However, since the direction of arrival θ _k cannot be actually observed, of course, if this is regarded as a latent variable _, the generation model of a _{k, ω} is a mixed model with DOA as a latent variable. If this can be incorporated into the above-described infinite number of sound source mixture model and parameter inference of the entire generation model can be performed under the observation signal, sound source separation and permutation matching may be solved simultaneously.

First, a set of I DOA candidate values consisting of Θ ₁ ,..., Θ _I (all constants) is prepared. For example, suppose that a set of angles Θ _i = (i−1) π / I, (i = 1,..., I) obtained by dividing 180 degrees into I equal parts. Assuming a process in which the DOA of each sound source is selected and determined from the DOA candidate values, the process for generating θ _k is as shown in the following equations (13) and (14): Can be described.

However, ρ _k ^ = (ρ _{k, 1} ,..., Ρ _{k, I} ). x _k ∈ {1, ..., I} is an indicator variable indicating which DOA candidate value is assigned to the k-th sound source, and the above equation (13) is a discrete distribution (each probability value is represented by ρ _{k, 1} , ..., ρ _{k, I} ). Through this process, the DOA of each sound source is determined, and the transfer frequency characteristic a _k (ω) is generated by the following (15).

Here, a Dirichlet distribution represented by the following equation (16) is assumed as a prior distribution of ρ _k ^.

The generation model of a _k (ω) based on the above mixed model is referred to as a “mixed DOA model”.

<Variation reasoning algorithm>
Given the observed signal y ^ = y ^ _{1: Ω, 1: T} , the parameters a ^ = a _{1: ∞, 1: Ω} ^, s ^ = s _{1: Ω, 1: T} , z ^ = z _{1: Ω, 1: T} , v ^ = v _{1: ∞} , x ^ = x _{1: ∞} , ρ ^ = ρ ^ _{1: K} posterior distribution p (a ^, s ^, z ^ , v ^, x ^, ρ ^ | y ^). Although it is difficult to obtain this posterior distribution analytically, an approximate distribution can be obtained by iterative calculation based on the variational reasoning method. Hereinafter, for simplicity, Σ ⁽ⁿ⁾ _{1: Ω} and Σ ^(a) _{1: Ω} , α ₀ , β ₀ are constants determined experimentally. Variational inference is based on the posterior distribution p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) and the non-negative variable function q (a ^, s The objective is to minimize the Kullback-Leibler divergence with respect to q between {circumflex over (z)}, {circumflex over (v)}, {circumflex over (v)}, {circumflex over (x)}, and {circumflex over (ρ)}.

However, <f (x)> _{q (x)} represents ∫q (x) f (x) dx. Assuming that q can be approximated by the following equation (19), q (a ^), q (s ^), q (z ^), q (v ^), q (x ^), q It is to obtain an approximate distribution of p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) by minimizing F [q] repeatedly for (ρ ^). This is the basic concept of variational reasoning.

  Also, with respect to q (z), as shown in the following equation (20), censored approximation is performed.

  This approximation does not mean that the model complexity (number of sound sources) is fixed, but that the function space of q is limited to a certain region. Therefore, if you want to fit p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) that you want to estimate with q as much as possible, the larger D, the better. . Although derivation is omitted, each q that minimizes the above equation (18) under the constraint of the above equation (17) is analytically obtained as a form shown in the following equations (21) to (26).

M _{k, ω} ^, Γ _{k, ω} , μ _{ω, t} , σ ² _{ω, t} , φ _{ω, t} ^ = (φ _{1, ω, t} , _... , Φ _{D, ω, t} ), _{γ k, 0, γ k,} 1, ψ k ^ = (ψ k, 1,..., ψ k, I), ζ k, 1,..., ζ k, I , the following (27) It represents with Formula-(37) Formula. However,

And

Where Ψ is a digamma function. Further, “ ^* ” attached to the symbol represents a complex conjugate. In the above update formulas for γ _{k, 0} and γ _{k, 1} (equations (36) and (37)), the second term is multiplied by v ₀ , which is equivalent to raising the prior distribution to the v ₀ power. However, this is a measure for preventing the influence of the prior distribution from becoming too small when the data information increases due to long audio data. In the experiment described later, v ₀ = 0.7 × Ω × T.

<System configuration>
Next, an example in which the present invention is applied to an acoustic signal analysis apparatus that analyzes acoustic signals obtained from M (M is an integer of 2 or more) microphones and separates them into unknown K sound source signals will be described. Thus, an embodiment of the present invention will be described.

  As shown in FIG. 1, the acoustic signal analysis apparatus according to the embodiment of the present invention is a computer that includes a CPU, a RAM, and a ROM that stores a program for executing an acoustic signal analysis processing routine described later. It is configured and functionally configured as follows.

  The acoustic signal analysis device 100 includes an input unit 10, a calculation unit 20, a storage unit 30, and an output unit 40.

The input unit 10 receives time-series data of acoustic signals (multi-channel signals) output from the M microphones. The storage unit 30 stores time-series data of acoustic signals input from the input unit 10. In addition, the storage unit 30 stores the results of each processing described later, and stores constant parameters Σ ⁽ⁿ⁾ _ω ^, Σ ^(a) _ω ^, α ₀ , β ₀ .

  For example, the number of microphones M is 2. The constant parameters are set as shown in the following equations (38) to (41), respectively.

Here, I ^ is an M × M unit matrix. The number of angle divisions is I = 180. v ₀ = 0.7 x Ω x T. Σ ⁽ⁿ⁾ _ω ^

Represents a y _ω, the allowable amount of deviation from the _t (observation signal) of the (sparse sound source ^model), Σ ^(a) _ω ^ _is, a _k, of _ω

This represents the allowable amount of deviation from the steering vector corresponding to the direction of arrival of the sound source k when it is known. Also, alpha ₀ represents the degree to which large index of the sound source is less likely to become active, beta ₀ DOA x distribution [rho _{k, 1} of _k, ..., represent the degree to which [rho _{k, I} becomes sparse.

  The calculation unit 20 includes a time frequency analysis unit 21, an initial setting unit 22, a parameter update unit 23, a parameter adjustment unit 24, an end determination unit 25, a time frequency component estimation unit 26, and a signal conversion unit 27. I have. The parameter updating unit 23 includes an active sound source posterior distribution updating unit 231, a time frequency component posterior distribution updating unit 232, a sound source indicator posterior probability updating unit 233, a sound source arrival direction posterior probability updating unit 234, and a steering vector posterior probability. An update unit 235 and a sound source direction posterior probability update unit 236 are provided.

The time frequency analysis unit 21 receives the observed acoustic signal as a time-series signal of each microphone, and inputs time frequency components (observation time frequency components) y _{m, ω, t} (m = 1,..., M, ω = 1,..., Ω, t = 1,..., T are indices corresponding to microphones, frequencies, and times, respectively)) in three dimensions (m, ω, t) Compute the array y ^. Further, the calculated time frequency component y _{m, ω, t} is stored in the storage unit 30. More specifically, the time-frequency analysis unit 21 performs time-frequency analysis on each microphone m by using time-series data of the acoustic signal of the microphone as an input and using a short-time Fourier transform (STFT). time-frequency components y _m by _{performing, omega,} computes the _t, time-frequency components y _{m, omega,} matrix that contains the _t (amplitude spectrogram) y ^ = (y m, ω, t) and _{M ×} Ω _{× T} Output. Note that the time frequency component ym _{, ω, t} may be calculated using wavelet transform.

The initial setting unit 22 sets initial values of Ω, l _{i, ω} ^ and parameters of each variational posterior distribution q (hereinafter referred to as variational parameters) as follows, and stores them in the storage unit 30.

First, Ω = Ω ₀ is set so that iterative calculation is started from a frequency band h _Ω0 = about 300 Hz to about 300 + 120 Hz that is high enough to cause DOA to appear in phase and in which aliasing does not occur.

For l _{i, ω} ^, an initial value is set for each (i, ω) according to the following equation (42).

The setting, average mu _omega active source _signals, for _t, the average sigma _m y _m in the time-frequency components of acquired signals by the microphones _{m, omega,} the _t / M as an initial value (m, t) for each To do.

In addition, φ _{k, ω, t} (parameter representing the probability that the sound source k is active at the time frequency (ω, t)) is inversely proportional to the sound source index (so that a larger index has a lower probability value) Moreover, it is given for each (k, m, t) according to the following equation (43) so that Σ _k φ _{k, ω, t} = 1.

In addition, other variational parameters _{m k, ω ^, Γ k} , ω, σ 2 ω, t, γ k, 0, γ k, 1, ψ k ^ = (ψ k, 1,..., Ψ k _{, I} ), ζ _{k, 1} ,,..., Ζ _{k, I} may be set to appropriate values using random numbers as initial values.

The time-frequency component posterior distribution updating unit 232 stores σ ² _{ω, t} , φ _{k, ω, t} , m _{k, ω} ^, stored in the storage unit 30 for each of all combinations of (ω, t). Based on Σ ⁽ⁿ⁾ _ω ^, y _{m, ω, t} , Γ _{k, ω} , the parameters _{ω ω,} of the posterior distribution of the time-frequency component of the active sound source according to the following equations (44) and (45) _t and σ ² _{ω, t} are updated and stored in the storage unit 30.

The steering vector posterior probability update unit 235 stores Γ _{k, ω} , σ ² _{ω, t} , φ _{k, ω, t} , m _k stored in the storage unit 30 for each of all combinations of (k, ω). _{, ω} ^, Σ ⁽ⁿ⁾ _ω ^, Σ ^(a) _ω ^, μ _{ω, t} , y _{m, ω, t} , ψ _k , l _{i, ω} ^ 47) The parameters m _{k, ω} ^, Γ _{k, ω} , of the a posteriori probability of the steering vector of each sound source are updated according to the equation (47) and stored in the storage unit 30.

The sound source indicator posterior probability update unit 233 is configured to store Γ _{k, ω} , σ ² _{ω, t} , φ _{k, ω, t} , stored in the storage unit 30 for each of all combinations of (k, ω, t). m _{k, ω} ^, Σ ⁽ⁿ⁾ _ω ^, μ _{ω, t} , y _{m, ω, t} , γ _{k, 0} , γ _{k, 1} The parameter φ _{k, ω, t} of the posterior probability of the indicator (which sound source is active in each time frequency bin) is updated and stored in the storage unit 30.

The sound source arrival direction posterior probability updating unit 234 adds ζ _{k, i} , Σ ^(a) _ω ^, l _{i, ω} ^, stored in the storage unit 30 for each of all combinations of (k, i). Based on the following equation (49), the parameter ψ _{k, i} of the posterior probability indicating from which direction each sound source has come is updated and stored in the storage unit 30.

The sound source direction posterior probability update unit 236 determines, for each of all combinations of (k, i), according to the following equation (50) based on ψ _{k, i} , β ₀ stored in the storage unit 30. The parameter ζ _{k, i} of the posterior probability that the sound source seems to be in the direction is updated and stored in the storage unit 30.

The active sound source posterior distribution updating unit 231 performs, for each k, based on φ _{k, ω, t} , α ₀ , v ₀ stored in the storage unit 30 according to the following equations (51) and (52): The parameters γ _{k, 0} , γ _{k, 1} of how much each sound source is likely to become active are updated and stored in the storage unit 30.

  The parameter adjustment unit 24 performs the following adjustment.

First, according to the following equation (53) _, the scale of the steering vector is obtained by normalizing the scale of m _{k, ω} ^ stored in the storage unit 30 for each combination of (k, ω). Normalize.

However, [·] _i represents the i-th element of the vector.

  Further, the frequency band used for the iterative calculation is expanded by the number of frequency bins corresponding to ΔΩ = about 120 Hz according to the following equation (54).

Further, the sound source index k is rearranged so that the frequency μ at which the sound source k becomes active and the average μ _{ω, t} of the time frequency components s _{ω, t} at time _t are in descending order.

For example, after the 20th iteration, the covariance matrix Σ ^(a) _ω of the steering vector is further reduced by (1 / number of iterations) for each ω.

  The end determination unit 25 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, the processes of the parameter update unit 23 and the parameter adjustment unit 24 are repeated. When it is determined that the end condition is satisfied, the end determination unit 25 proceeds to processing by the time frequency component estimation unit 26.

  As the termination condition, it may be used that the number of iterations has reached a predetermined number. Note that the fact that the change rate of the parameter by one parameter update can be regarded as almost 1 may be used as the termination condition. Further, the fact that the rate of change of the value of the objective function F [q] by one parameter update can be regarded as almost 1 may be used as the termination condition.

The time frequency component estimator 26 calculates the following (55) based on φ _{k, ω, t} and μ _{ω, t} stored in the storage 30 for each of all combinations of (k, ω, t). ) Calculate the estimated value ^ s _{k, ω, t} of the time frequency component of each sound source k according to the equation.

The signal conversion unit 27 converts the estimated value s _{k, ω, t} of the time frequency component of each sound source k calculated by the time frequency component estimation unit 26 into the sound source signal of each sound source, and the output unit 40 causes the sound source to Output a signal.

<Operation of acoustic signal analyzer>
Next, the operation of the acoustic signal analysis apparatus 100 according to the present embodiment will be described. First, time-series data of acoustic signals from each microphone is input to the acoustic signal analyzing apparatus 100 as a signal to be analyzed and stored in the storage unit 30. Then, in the acoustic signal analysis device, an acoustic signal analysis processing routine shown in FIG. 2 is executed.

First, in step S101, for each microphone, an acoustic signal in a frame at each time t is read from the storage unit 30, and a time frequency analysis using a short-time Fourier transform is performed on the acoustic signal. A three-dimensional array y ^ having observation time frequency components y _{m, ω, t} as elements of each (m, ω, t) is generated and stored in the storage unit 30.

In step S102, Ω, l _{i, ω} ^ and parameters μ _{ω, t} , φ _{k, ω, t} , m _{k, ω} ^, Γ _{k, ω} , σ ² _{ω, t of} each variational posterior distribution q _{, γ k, 0, γ k} , 1, ψ k ^ = (ψ k, 1,..., ψ k, I), ζ k, 1,..., the initial values of the zeta _{k, I} And stored in the storage unit 30.

In the next step S103, the parameters σ ² _{ω, t} , φ _{k, ω, t} , m _{k, ω} ^, Γ _{k, ω} set in step S102 or updated in steps S103, S104, S105, and Based on _{ym, ω, t} obtained in step S101, the parameters _{ω ω, t} , σ ² _ω of the posterior distribution of the time frequency component of the active sound source according to the above formulas (44) and (45). _{, t} are updated and stored in the storage unit 30.

In step S104, the parameters Γ _{k, ω} , σ ² _{ω, t} , φ _{k, ω, t} , m _{k, ω} ^ set in step S102 or updated in steps S103, S104, S105, S106 are _obtained. , Μ _{ω, t} , ψ _k , l _{i, ω} ^ and ym _{, ω, t} obtained in step S101, and according to the above equations (46) and (47), the steering of each sound source The vector posterior probability parameters m _{k, ω} ^ and Γ _{k, ω} are updated and stored in the storage unit 30.

In step S105, the parameters Γ _{k, ω} , σ ² _{ω, t} , φ _{k, ω, t} , m _{k, ω} ^, μ set in step S102 or updated in steps S103, S104, S105, and S108. _{Based on ω, t} , γ _{k, 0} , γ _{k, 1} and ym _{, ω, t} obtained in step S101, an active sound source indicator (each time frequency bin) according to the above equation (48). The a posteriori probability parameter φ _{k, ω, t} of which sound source is active in () is updated and stored in the storage unit 30.

In the next step S106, based on the parameters ζ _{k, i} , l _{i, ω} ^ set in step S102 or updated in step S107, each sound source has come from which direction according to the above equation (49). The posterior probability parameter ψ _{k, i} is updated and stored in the storage unit 30.

In the next step S107, based on the parameter ψ _{k, i} set in step S102 or updated in step S106, the posterior probability parameter ζ in which direction the sound source is likely to exist according to the above equation (50). _{k and i} are updated and stored in the storage unit 30.

In step S108, how active each sound source is likely to become active according to the equations (51) and (52) based on the parameters φ _{k, ω, t} set in step S102 or updated in step S105. The posterior distribution parameters γ _{k, 0} , γ _{k, 1} are updated and stored in the storage unit 30.

In step S109, the parameter m _{k, ω} ^ updated in step 104 is normalized, the parameter Ω is updated, and the sound source index k is rearranged. In addition, the parameter Σ ^(a) _ω is adjusted after the 20th iteration.

In the next step S110, as an end condition, it is determined whether or not the number of iterations has reached a predetermined number of times. If the number of iterations has not reached the predetermined number of times, the end condition is satisfied. If it is determined that there is not, the process returns to step 103, and the processes of steps 103 to 109 are repeated. On the other hand, if the number of iterations reaches a predetermined number of times, it is determined that the termination condition is satisfied, and in step S111, φ _{k, ω, t} , _μω, Based on _t , an estimated value s s _{k, ω, t} of the time frequency component of each sound source k is calculated according to the above equation (55).

In step S112, the sound source signal of each sound source is calculated based on the estimated value s _{k, ω, t} of the time frequency component of each sound source k calculated in step S111 and output by the output unit 40. Then, the acoustic signal analysis processing routine is finished.

<Experimental result>

Next, in order to show the usefulness of the method according to the present embodiment (hereinafter referred to as the proposed method), the result of a verification experiment of the sound source separation performance will be described. Audio signals of three speakers (two women and one man) (references (A. Kurematsu, K. Takeda, Y. Sagisaka, S. Katagiri, H. Kuwabara, and K. Shikano, “ATR Japanese speech database as a tool of speech recognition and synthesis, ”Trans. Speech Communication, pp. 357-363, 1990.), room impulse response (reverberation time is 0 ms) (reference (S. Nakamura, K. Hiyane , F. Asano, T. Nishiura, and T. Yamada, “Acoustical sound database in real environments for sound scene understanding and hands-free speech recognition,” in Proc. LREC '00, 2000, pp. 965-968.) The signal that was artificially mixed by convolutional addition was used as the observation signal. The sampling frequency was 16 kHz. The time frequency component of the observation signal was calculated by short-time Fourier transform (frame length is 64 ms, frame shift is 16 ms). Σ (n) ω and Σ (a) ω, respectively ^ I, was a 10- ^1.5 × ^ I. The number of angle divisions was I = 180. After executing the above iterative algorithm, the estimated value of the active sound source component μω, t is multiplied by the probability value φk, ω, t representing how active the sound source k is in each time frequency bin, The estimated time-frequency component. In this experiment, considering the possibility that ζ falls into a local solution due to spatial aliasing, the algorithm is executed using only low-band observation information where spatial aliasing does not occur in the initial stage of iterative calculation. As the number of times increased, the band was gradually expanded to a higher band.

  Signal-to-Distortion Ratio (SDR) was adopted as an evaluation standard for sound source separation performance (references (E. Vincent, R. Gribonval, and C. F´evotte, “Performance measurement in blind audio source separation,” IEEE Trans. ASLP, pp. 1462-1469, 2006.)). First, in order to verify the effect of adaptively inferring the required number of sound sources, which is the first point of the proposed method, we confirmed how the separation performance is affected by the truncation level D. In addition, it was necessary to assume the number of sound sources in the conventional method, but we also confirmed how the separation performance by the conventional method changes depending on the assumed number of sound sources K. The result is shown in FIG. In the conventional method, the separation performance was high when the actual number of sound sources and the assumed number of sound sources were the same, but when the numbers were not the same, the separation performance was extremely low. It can be confirmed that the larger D is, the better the approximation of true posterior distribution is obtained for q, so that the separation performance improves as D increases. This means that if D is set large in advance, high separation performance can be obtained even if the number of sound sources is not known, and the effect of the proposed method is shown.

Next, in order to see the effect of the mixed DOA model, it was confirmed whether the direction of arrival of the sound source could be estimated correctly. Fig. 4 is a plot of the inter-channel phase difference arg ([m _{k, ω} ^] ₂ / [m _{k, ω} ^] ₁ ) of the indoor impulse response used for the synthesis of the observed signal in different colors for each sound source. Yes, FIG. 5 plots the inter-channel phase difference arg ([m _{k, ω} ] ₂ / [m _{k, ω} ] ₁ ) calculated from the estimated transfer frequency characteristics m _{k, ω} ^ in different colors for each sound source. It is what. However, [•] i represents the i-th element of the vector. It can be seen that even if there is spatial aliasing, the direction of arrival of each sound source can be estimated roughly.

As described above, according to the acoustic signal analysis device according to the embodiment of the present invention, there is one sound source that is active at each time t for each frequency ω, and the probability v _k that each sound source k becomes active. The prior distribution is modeled by a bar folding process, the prior distribution of the indicator x _k indicating the arrival direction index of each sound source k is modeled by a discrete probability distribution, and the probability ρ _k ^ of each sound source k being the arrival direction of each index The prior distribution is modeled as a directory distribution, and the parameters of the variation function q that approximates the posterior distribution of the parameters of the sound source signal generation model are repeatedly updated based on the variational inference method, and the time-frequency component of the sound source signal of each sound source k is updated. By estimating the posterior distribution, even if the number of sound sources is unknown, the time-series data of acoustic signals output from multiple microphones can be used to accurately generate sound sources for each sound source. It is possible to Ku separation.

  In addition, since individual sound source signals mixed from acoustic signals acquired by a plurality of microphones can be separated, applications such as a hands-free video conference system and a system for automatically creating conference contents are expected.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above-described acoustic signal analysis apparatus has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 21 Time frequency analysis part 22 Initial setting part 23 Parameter update part 24 Parameter adjustment part 25 End determination part 26 Time frequency component estimation part 27 Signal conversion part 30 Storage part 40 Output part 100 Acoustic signal analysis apparatus 231 Active Sound source posterior distribution update unit 232 Time frequency component posterior distribution update unit 233 Sound source indicator posterior probability update unit 234 Sound source arrival direction posterior probability update unit 235 Steering vector posterior probability update unit 236 Sound source direction posterior probability update unit

Claims (6)

Using time series data of acoustic signals output from M microphones (M is an integer of 2 or more) as input, the observation time frequency components y _{m, ω, t} (m is a microphone, t is time, ω is frequency) A time-frequency analysis means for outputting a three-dimensional array y ^ having an element as an index;
For each frequency ω, it is included in a three-dimensional array a ^ having elements of transmission frequency characteristics a _{m, k, ω} of sound source signals from each of D sound sources k (D is an integer of 1 or more) to each microphone m. Parameters of posterior distribution of each vector a _{k, ω} ^ (mean vector m _{k, ω} ^, covariance matrix Γ _{k, ω} ) of complex normal distribution, time frequency component s of active sound source at each time t for each frequency ω Parameters of the posterior distribution of each element s _{ω, t} included in the two-dimensional array s ^ having _{ω, t} as elements (mean μ _{ω, t} of the complex normal distribution, variance σ ² _{ω, t} ), and each frequency ω Parameters of the posterior distribution of each element z _{ω, t} of the two-dimensional array z ^ having an indicator z _{ω, t} indicating the index of the sound source active at time t (each parameter φ _{k, ω, t of the} discrete probability distribution) ), vector with a probability v _k that each sound source k becomes active element v ^ parameters posterior distribution of each element v _k of (parameter gamma _k of the beta _{distribution, 0, γ k, 1)} , the vector x ^ of having the indicator x _k of the index i in the arrival direction of each sound source k elements Parameters of the posterior distribution of each element x _k (each parameter ψ _{k, i of the} discrete probability distribution) and each of the two-dimensional arrays ρ ^ having the probability ρ _{k, i} that each sound source k is the arrival direction of each index _i as elements Initial setting means for setting an initial value of each parameter of the posterior distribution of the vector ρ _k ^ (each parameter ζ _{k, i of the} directory distribution);
Given the three-dimensional array y ^, the three-dimensional array a ^, the two-dimensional array s ^, the two-dimensional array z ^, the vector v ^, the vector x ^, and the two-dimensional array ρ ^ Posterior distribution p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) and a variable function q (a ^, s ^, z ^, v ^, x ^, ρ ^) The average vector m _{k, ω in} each of all the combinations of (k, m) so that the objective function is minimized based on the variational reasoning method, with the function representing the divergence representing the difference between ^ And the variance Γ _{k, ω} and the average μ _{ω, t} and the variance σ ² _{ω, t} in each of all combinations of (ω, t) and all combinations of (k, ω, t). wherein in each parameter phi _{k, omega,} and _t, the in all k parameter _{γ k, 0, γ k,} 1, and the parameter ψ _{k, i, (k,} And parameter updating means for updating the parameter zeta _{k, i} in each of all combinations of)
An end determination means for repeatedly updating by the parameter update means until a predetermined end condition is satisfied;
For each combination of (k, ω, t), the time frequency components s _{k, ω, t} of the sound source signal of the sound source k are calculated based on the parameters φ _{k, ω, t} and the average μ _{ω, t.} A sound source signal estimating means for estimating;
An acoustic signal analyzing apparatus including: The variable function q (a ^, s ^, z ^, v ^, x ^, ρ ^) is changed to q (a ^) q (s ^) q () q (z ^) q (v ^) q ( x ^) q (ρ ^)
Q (a ^) that minimizes the objective function is multiplied by the complex normal distribution of the vectors a _{k, ω} ^ in each of all combinations of (k, ω),
Q (s ^) that minimizes the objective function is multiplied by the complex normal distribution of the elements s _{ω, t} in each of all combinations of (ω, t),
Q (z ^) that minimizes the objective function is multiplied by the discrete probability distribution of the element z _{ω, t} in each of all combinations of (ω, t),
Q (v ^) that minimizes the objective function is multiplied by the beta distribution of the element v _{k in} each of all k,
Q (x ^) that minimizes the objective function is multiplied by the discrete probability distribution of the element x _{k in} each of all k,
2. The acoustic signal analyzing apparatus according to claim 1, wherein q (ρ ^) that minimizes the objective function is multiplied by a directory distribution of the vector ρ ^ _k in each of all k. Predetermine the number D of truncation of the number of sound sources
The acoustic signal analyzing apparatus according to claim 2 _{, wherein} q (z _{ω, t} = k ') = 0 is set for k' _{that is equal to} or greater than D + 1. Normalizing the scale of the average vector m _{k, ω} in each of all the combinations of (k, m) updated by the parameter updating means, and for the index k of the sound source, the frequency ω, at which the sound source k becomes active Parameter adjusting means for rearranging the index k so that the average _{μω, t at} time t is in descending order;
The acoustic signal analyzer according to claim 1, wherein the end determination unit repeatedly performs the update by the parameter update unit and the process by the parameter adjustment unit until the predetermined end condition is satisfied. Time-frequency analysis means ym _{, ω, t} (where m is a microphone and t is a microphone) with time-series data of acoustic signals output from M (m is an integer of 2 or more) microphones m as input. Time, ω is the frequency index.)
By the initial setting means, for each frequency ω, a three-dimensional array having the transmission frequency characteristics a _{m, k, ω} of the sound source signal from each of the D sound sources k (D is an integer of 1 or more) to each microphone m Parameters of a posteriori distribution of each vector a _{k, ω} ^ included in a ^ (mean vector m _{k, ω} ^, covariance matrix Γ _{k, ω} ) of a complex normal distribution, active sound source at each time t for each frequency ω Parameters of the posterior distribution of each element s _{ω, t} included in the two-dimensional array s ^ having the time frequency component s _{ω, t} of the element (mean μ _{ω, t} of the complex normal distribution, variance σ ² _{ω, t} ), Parameters of the posterior distribution of each element z _{ω, t} of the two-dimensional array z ^ having an indicator z _{ω, t} indicating the index of the sound source that is active at each time t for each frequency ω (each parameter φ of the discrete probability distribution) _{k, ω, t),} the probability of each sound source k becomes active vector with _k elements v ^ parameters of the posterior distribution of each element v _k of (beta distribution parameter _{γ k, 0, γ k,} 1), indicator x _k elements indicating the direction of arrival of the index i of each sound source k Posterior distribution parameter (each parameter ψ _{k, i of} discrete probability distribution) of each element x _k of vector x ^, and the probability ρ _{k, i} that each sound source k becomes the arrival direction of each index i. Set the initial values of the posterior distribution parameters (directive distribution parameters ζ _{k, i} ) of each vector ρ _k ^ in the dimensional array ρ ^
When the three-dimensional array y ^ is given by the parameter updating means, the three-dimensional array a ^, the two-dimensional array s ^, the two-dimensional array z ^, the vector v ^, the vector x ^, and the A posteriori distribution p (a ^, s ^, z ^, v ^, x ^, ρ ^ | y ^) of the two-dimensional array ρ ^ and a variable function q (a ^, s ^, z ^, v ^, x ^) , Ρ ^) as a function representing the divergence between the objective function and the average in each combination of (k, m) so as to minimize the objective function based on the variational inference method. The mean μ _{ω, t} and the variance σ ² _{ω, t} in each of all combinations of the vector m _{k, ω} ^ and the variance Γ _{k, ω} and (ω, t) _, and (k, ω, t) of the parameter phi _k in each of all _{combinations, omega, t} and the parameter gamma _{k, 0} in all k, gamma _{k, 1,} And serial parameters [psi _k, updating the parameter zeta _{k, i} in each of all combinations of (k, i),
The updating by the parameter updating unit is repeatedly performed until a predetermined end condition is satisfied by the end determination unit,
The time-frequency component s of the sound source signal of the sound source k is determined by the sound source signal estimation means based on the parameters φ _{k, ω, t} and the average μ _{ω, t} for each of all combinations of (k, ω, t). An acoustic signal analysis method for estimating _{k, ω, t} .   The program for functioning a computer as each means of the acoustic signal analyzer of any one of Claims 1-4.