JP6193823B2 - Sound source number estimation device, sound source number estimation method, and sound source number estimation program - Google Patents

Description

  The present invention relates to a sound source number estimation device, a sound source number estimation method, and a sound source number estimation program.

  Conventionally, a technique for performing sound source separation based on clustering is known as a sound source separation technique. For example, assuming that the number of sound sources is known, clustering is performed using the same number of clusters as the number of sound sources (see, for example, Non-Patent Document 1). Since the feature values of one sound source tend to concentrate in the same feature value space within each frequency, sound source separation can be performed by such clustering.

  The shape of the feature amount space of each sound source depends on the frequency, but one sound source signal is likely to rise at the same time and fall easily at all frequencies. That is, sound source activity is synchronized between frequencies. In the clustering described in Non-Patent Document 1, the synchronism of the sound source activity is appropriately modeled and incorporated into the overall optimization, thereby realizing simultaneous clustering of the sound source position feature quantity and the sound source activity.

  However, the clustering described in Non-Patent Document 1 assumes that the number of sound sources is known. If the number of sound sources is unknown, clustering cannot be performed using the same number of clusters as the number of sound sources. On the other hand, a technique for estimating the number of sound sources is known. For example, as a technique for estimating the number of sound sources, there is a method for estimating the number of sound sources on the premise that “the number of sound sources ≦ the number of microphones and no reverberation” (see, for example, Non-Patent Document 2).

Nobutaka Ito, Akiko Araki, Tomohiro Nakatani “Clustering-based sound source separation without permutation problem based on time-varying mixture weights”, The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, VOL.113, NO.27, 20135 Moon MATI WAX, THOMAS KAILATH, “Detection of Signals by Information Theoretic Criteria”, IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL.ASSP-33, NO.2, APRIL 1985

  However, the conventional techniques for estimating the number of sound sources are based on the premise that “the number of sound sources ≦ the number of microphones and no reverberation”, and under conditions close to the actual environment, the number of sound sources can be estimated appropriately. There was a problem that it was not possible. In other words, the conventional technique for estimating the number of sound sources can only be realized under the ideal condition of “the number of sound sources ≦ the number of microphones and no reverberation”, for example, “the number of sound sources> the number of microphones, or there is reverberation”. If this is the situation, the number of sound sources cannot be estimated appropriately.

In order to solve the above-described problems and achieve the object, the sound source number estimation apparatus of the present invention uses k as the cluster index, τ as the time frame index, ω as the angular frequency, and N sound sources. A feature extraction unit that extracts a feature vector x _τω corresponding to an observation signal vector y _τω composed of a time-frequency component of each observation signal from an observation signal obtained by observing a mixed signal in which signals are mixed with M microphones; The vector x _τω is applied to a predetermined probability model, and the model parameters of the probability model are estimated using an evaluation function that gives a higher evaluation value as the time series of likelihood of each sound source is synchronized between frequency bins, using the model parameters, the observed signal vector y _Tauomega calculates the posterior probability of the conditional probability belonging to the N respective clusters configured more than sound mode Includes a Le estimating unit, the number of clusters probability after previous article takes a significant value, and the number of sound sources estimating unit for estimating a number of sound sources, and the probabilistic model, the weights of the distribution of the feature vectors x _Tauomega for each sound source A mixture model represented by a sum, wherein the mixture weight of the probability model depends on the time frame τ and does not depend on the angular frequency ω, and the model parameter of the probability model includes the mixture weight, It is a parameter of the distribution of the feature vector x _τω related to the sound source.

  According to the present invention, there is an effect that it is possible to appropriately estimate the number of sound sources even under conditions close to a real environment.

FIG. 1 is a diagram illustrating a functional configuration of the model estimation apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a processing flow of the model estimation apparatus according to the first embodiment. FIG. 3 is a diagram illustrating a functional configuration of the sound source number estimation apparatus according to the second embodiment. FIG. 4 is a diagram illustrating a processing flow of the sound source number estimation apparatus according to the second embodiment. FIG. 5 is a diagram illustrating an example of the posterior probability when the number of clusters exceeding the actual number of sound sources is set. FIG. 6 is a diagram showing experimental results. FIG. 7 is a diagram illustrating sound source position feature amounts plotted in the feature amount space when the sound source and the number of clusters are set to be the same. FIG. 8 is a diagram illustrating an example of the sound source position feature amount plotted in the feature amount space when the number of clusters is set larger than that of the sound source. FIG. 9 is a diagram illustrating an example of the sound source position feature amount plotted in the feature amount space when the number of clusters is set larger than that of the sound source. FIG. 10 is a diagram illustrating a functional configuration of the sound source number estimation apparatus according to the third embodiment. FIG. 11 is a diagram illustrating a processing flow of the sound source number estimation apparatus according to the third embodiment. FIG. 12 is a diagram illustrating a computer that executes the sound source number estimation method.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[Points of Invention]
The technical point of the sound source number estimation apparatus according to the present invention is that the number of sound sources can be estimated under conditions where the number of sound sources is unknown. Although details will be described later, clustering described in Non-Patent Document 1 is performed by setting the number of clusters larger than the number of sound sources, the posterior probability of each cluster is calculated, and the number of clusters for which the posterior probability takes a significant value, This is estimated as the number of sound sources.

[First embodiment]
1st Embodiment of this invention is a model estimation apparatus which observes the signal from a several sound source with a several microphone, and estimates the model parameter regarding each sound source.

  With reference to FIG. 1, the example of a function structure of the model estimation apparatus A of 1st embodiment is demonstrated. The model estimation apparatus A includes a frequency domain conversion unit 1, a feature extraction unit 2, and a model estimation unit 3. The model estimation unit 3 includes a posterior probability calculation unit 31, a parameter update unit 32, and a parameter holding unit 33. The parameter update unit 33 includes a mixture weight update unit 321, a correlation matrix update unit 322, an average direction update unit 323, a density parameter update unit 324, and a permutation solution unit 325.

With reference to FIG. 2, the operation example of the model estimation apparatus A will be described in the order of procedures. Mixed signals ~ y _t of the observed time domain by M microphones to frequency domain transform unit 1 is input. The time domain mixed signal ~ y _t is defined by equation (1).

Where t is the time index, • ^T (superscript T) is the transpose of the vector • ~ y _mt is the time domain mixture observed by the mth (1 ≦ m ≦ M) microphone Represents a signal.

The frequency domain transform unit 1 generates and outputs an observation signal vector y _τω in the time frequency domain from the input time domain mixed signal ~ y _t by short-time Fourier transform or the like (step S1). The observation signal vector _{yτω in} the time-frequency domain is defined by equation (2).

Here, tau represents the time frame index, the ω represents an angular frequency, y _Emutauomega is expressed in the time frequency domain mixed signals ~ y _mt.

The feature extraction unit 2 receives the time-frequency domain observation signal vector y _τω output from the frequency domain transformation unit 1 as input, and calculates and outputs a feature vector x _τω (step S2). Computation of a feature vector x _Tauomega may be performed by normalizing the observed signal vector y _Tauomega the time-frequency domain, performed by normalizing the observed signal vector y _Tauomega the time-frequency domain after the whitened _{Alternatively} , the observation signal vector y _τω in the time frequency domain may be normalized and then whitened and then normalized again. For example, when the feature vector x _τω is calculated by normalizing the observation signal vector y _τω in the time-frequency domain, the feature vector x _τω may be calculated by Expression (3).

Further, for example, when normalizing the observation signal vector y _τω in the time frequency domain after whitening, the feature vector x _τω may be calculated as _follows . First, the sample correlation matrix R _ω ^y of the observation signal vector y _{τω in} the time-frequency domain is calculated by the equation (4) using the observation signal vector y _τω in the time-frequency domain.

Here, T is the number of frames, and • ^H (superscript H) is Hermite transpose.

Next, eigenvalues and eigenvectors of the sample correlation matrix R _ω ^y are calculated. The eigenvalues of the calculated sample correlation matrix R _ω ^y arranged in descending order are represented as σ _ω1 , σ _ω2 ,..., Σ _ωM . Therefore, the relationship of Formula (5) is established.

Here, since the sample correlation matrix R ω ^_y is a Hermitian matrix, the eigenvalues _{σ ω1, σ ω2, ...,} σ ωM is to note that all is a real number. Further, the eigenvectors of the sample correlation matrix R _ω ^y forming the orthonormal system corresponding to the eigenvalues σ _ω1 , σ _ω2 ,..., Σ _ωM are represented by u _ω1 , u _{ω2 ,.} Here, since the sample correlation matrix R _ω ^y is a Hermitian matrix, it should be noted that such an eigenvector exists.

Next, matrix sigma _omega determined by equation (6), the matrix U _omega is obtained by equation (7).

Next, a vector y ′ _τω obtained by whitening the observation signal vector y _τω in the time-frequency domain using the matrix U _ω and the matrix Σ _ω is calculated according to the equation (8).

Finally, the feature vector x _τω is calculated by normalizing the vector y ′ _τω with its norm as in the following equation.

The model estimation unit 3 applies the feature vector x _τω to a probability model representing the distribution of the feature vector, and calculates a model parameter of the probability model suitable for clustering using a predetermined evaluation function for evaluating the probability model. Then, the model estimation unit 3 calculates a posteriori probability, which is a conditional probability belonging to each cluster in which the observed signal vector _yτω is set to be larger than N sound sources, using the model parameter.

When the position of the sound source is fixed, in each frequency bin, the feature vector x _τω ideally takes a unique value for each sound source. However, in practice, there are fluctuations due to the effects of noise and reverberation, modeling errors, and the like, so the feature vector x _τω forms a cluster around a certain value for each sound source. Therefore, in the present invention, the distribution of the feature vector x _τω related to the cluster k is modeled by, for example, the Watson distribution as follows.

Where a _kω represents the center of the distribution of feature vectors for cluster k and is called the mean orientation, κ _kω represents the small spread of the distribution of feature vectors for cluster k, and the density parameter (concentration parameter). M (a, b, x) is a Kummer function. For details on the Kummer function, see “S. Sra and D. Karp,“ The multivariate Watson distribution: maximum-likelihood estimation and other aspects ”, arXiv: 1104.4422v2, 2012. (Reference 1)”. Note that the distribution of feature vectors is defined for each frequency bin.

Many sound source signals including speech have a common amplitude modulation property that “the time series {| s _nτω |} _τ of the amplitude value of the time-frequency conversion of the sound source signal is similar between frequency bins” (for example, , “GJ Brown,“ Computational Auditory Scene Analysis: A Representational Approach ”, Ph.D. thesis, University of Sheffield, 1992.”). In the present invention, attention is paid to the fact that the property of the common amplitude modulation can be used as a clue to avoid the permutation problem. Based on the assumption that there is at most one sound source signal contributing to each time frequency component of the mixed signal (WDO (W-Disjoint Orthogonality) property), this common amplitude modulation property is easy to use in the framework of clustering In other words, it can be said that “the time series {d (τ, ω)} _τ of cluster numbers contributing to the observation signal is similar between frequency bins”. In the present invention, the similarity of {d (τ, ω)} _τ between the frequency bins is expressed as “the prior distribution P (d (τ, ω) = k) of d (τ, ω) in the frame τ. It depends (time-varying) and does not depend on the frequency bin (angular frequency ω) (frequency-independent) ”. By using the commonality of amplitude modulation between frequency bins for each sound source signal, clustering can be performed without causing permutation. This prior probability is represented by α _kτ . Α _kτ satisfies Σ _{k = 1} ^K α _kτ = 1. Here, it is assumed that the number of sound sources is unknown, and it is assumed that the number of clusters exceeding the number of sound sources is set in the model estimation apparatus A.

This prior probability may be assumed to change every one time frame, or may be assumed to change every block consisting of a plurality of time frames. Assuming that the prior probability changes every time frame, for any cluster k and any time frame τ, α _kτ is an independent variable and a parameter to be estimated.

On the other hand, if it is assumed that the prior probability changes for each block consisting of several time frames, B is the total number of blocks, block numbers are b = 1, 2,..., B, and J is a time frame within each block. If the time frame number in each block is j = 1, 2,..., J, it can be expressed as τ = (b−1) × J + j, α _{k, (b−1) × J + j (j = 1,2, ...,} J) from the equal, mixture weights are parameters to be estimated is ~ α _kb = α _k, with ~ alpha _kb defined by _{(b-1) × J +} 1 is there. In the following, a case will be described where it is assumed that the prior probability changes every one time frame unless otherwise specified.

From the above, the likelihood function of the feature vector x _τω is given by the mixed model expressed by the equation (11).

Here, Θ is a parameter set shown in Expression (12).

Here, {α _kτ } _kτ is defined by equation (13).

Other similar notations are defined accordingly. Hereinafter, α _kτ is referred to as a mixing weight. As a prior distribution of the mixture weight α _kτ , a Dirichlet distribution shown in Expression (14) is used.

Here, Γ is a gamma function, and φ is called a hyperparameter. By setting the value of φ sufficiently large, fluctuations in the mixing weight α _kτ can be suppressed. Although it is not necessary to finely adjust the value of φ, for example, values such as φ = 1, 10, 100, 1000 can be used.

A uniform prior distribution is assumed for parameters other than the mixing weight α _kτ . Therefore, it is _{p (Θ) = Π τ p} ({α kτ} k).

The model estimation unit 3 applies the feature vector x _τω to the probability model modeled as described above, and obtains a parameter set Θ suitable for posterior probability and clustering using a predetermined evaluation function for evaluating the probability model.

Hereinafter, the process of each part of the model estimation part 3 is demonstrated in detail. The model estimation unit 3 includes a posterior probability calculation unit 31, a parameter update unit 32, and a parameter holding unit 33, as shown in FIG. Prior to processing in the model estimation unit 3, initial values of the parameter set Θ are prepared in the parameter holding unit 33 (step S0). The initial value is, for _example, α kτ = 1 / K, and _κ kω = 20, a kω can be set by selecting at random from _{x τω} τω.

The posterior probability calculation unit 31 uses the parameter set Θ stored in the parameter holding unit 33 to _give a posterior probability γ _kτω , that is, a conditional probability that d (τ, ω) = k when the feature vector x _τω is given by the formula ( 15) (step S31).

  As shown in FIG. 1, the parameter update unit 32 includes a mixture weight update unit 321, a correlation matrix update unit 322, an average direction update unit 323, a density parameter update unit 324, and a permutation solution unit 325, and a current parameter set Θ Is updated to generate a new parameter set Θ ′ (step S32).

Mixing weight updating unit 321 uses the posterior probability gamma _Keitauomega, by calculating equation (16), and updates the mixture weight alpha _Lkr new value alpha _'Lkr.

Here, F represents the number of frequency bins. When _φ = 1, α 'kτ it can be seen that the average value of the posterior probability gamma _Keitauomega over all frequency bins. As φ increases, α ′ _kτ approaches the constant 1 / K.

The correlation matrix updating unit 322 updates the correlation matrix R _kω for each cluster k to a new value R ′ _kω by calculating Expression (17) using the feature vector x _τω and the posterior probability γ _kτω .

The average direction updating unit 323 updates the average direction a _kω to a new value a ′ _kω as a normalized principal component vector of the correlation matrix R _kω .

The density parameter updating unit 324 updates the density parameter к _kω to a new value к ′ _kω using Equation (18) using the maximum eigenvalue λ _kω of the correlation matrix R _kω .

As shown in equations (19) to (21), the permutation solving means 325 uses the average direction a ′ _kω and the density parameter к ′ _kω as the posterior probabilities p (Θ ′ | {x _τω } for each frequency bin. _The permutation is solved by replacing between sound sources so that _τω ) is maximized (step S325).

In the above, the processing when the mixing weight changes for each time frame has been described. However, when the mixing weight changes for each block composed of a plurality of time frames, the mixing weight in the mixing weight update unit 321 is used. in alpha _Lkr update equations (16), replacing the sum in the time frame of the posterior probability gamma _Keitauomega molecules to the sum of the block of the posterior probability gamma _Keitauomega, the F in the denominator may be replaced by F × J. On the other hand, the correlation matrix updating unit 322, the average direction updating unit 323, the density parameter updating unit 324, and the permutation solving unit 325 perform the same processing as that when the mixing weight changes for each time frame. Just do it.

Hereinafter, the basis for deriving each update formula in the parameter update unit 32 will be described. Parameter update is performed based on the EM (Expectation-Maximization) algorithm. Note that {d (τ, ω)} _τω is treated as a hidden variable in the EM algorithm.

First, a cost function L (Θ) for MAP (Maximum a posteriori) estimation is given by equations (22) to (24).

Here, {x _τω } _τω is assumed to be independent from each other, and a constant term independent of Θ is ignored. This objective function is maximized under the constraint condition shown in equation (25).

Since the objective function L (Θ) takes a large value when there is no permutation problem, the permutation problem can be avoided by maximizing L (Θ). In fact, as can be seen from the first term of equation (24), the objective function L (Θ) increases because of the likelihood (probability) of cluster k with respect to k and τ where the mixture weight α _kτ takes a large value. ) When p ( _xτω | d (τ, ω) = k, a _kω , κ _kω ) is large. Therefore, by maximizing L (Θ), the time series of likelihood {p (x _τω | d (τ, ω) = k, a _kω , κ _kω )} _τ for cluster k is synchronized between frequency bins. Considering this and the above-mentioned property that “the time series of the sound source index contributing to the observation signal {d (τ, ω)} _τ is similar between frequency bins”, L (Θ) is permutation. It can be seen that it takes a large value when there is no problem. The evaluation function (Q function) used in the EM algorithm is given by equations (26) and (27).

The updated parameter set Θ ′ is defined by the following equation.

ここで、μはラグランジュの未定乗数である。 It is derived that the Q function is maximized under the constraint of equation (25). That is, Expression (16) for _obtaining a new value α ′ _kτ of the mixture weight α _kτ is derived from Expressions (28) and (29) by Lagrange's undetermined multiplier method.

Here, μ is Lagrange's undetermined multiplier.

を解くことで得られる。したがって、クーラン・フィッシャー（Courant-Fischer）の定理より、R_kωの最大固有値に対応する固有ベクトル（主成分ベクトル）によって、平均方向を更新すればよい。 The average direction update formula is

Can be obtained by solving Therefore, the average direction may be updated by the eigenvector (principal component vector) corresponding to the maximum eigenvalue of R _{kω according to} the Courant-Fischer theorem.

As for the density parameter update formula (18), formula (31) is first obtained from ∂Q / ∂κ _kω = 0.

であり、λ_kωは相関行列R_kωの最大固有値である。上式は、近似的に次の式(３２)のように解くことができる。 here,

Λ _kω is the maximum eigenvalue of the correlation matrix R _kω . The above equation can be approximately solved as the following equation (32).

  The parameter holding unit 33 stores the parameter set Θ ′ obtained by the update process in the parameter update unit 32 (step S33). In the next processing by the posterior probability calculation unit 31, the stored parameter set Θ 'is provided as a parameter set. By using the parameter set Θ estimated using the model estimation apparatus of the first embodiment, sound source localization, sound source separation under conditions where the number of sound sources is unknown, simultaneous estimation of the number of sound sources and separated sounds, etc. Is possible.

The processing from step S31 to step S33 is repeated until the preset maximum number of iterations max_iter is reached or until the fluctuation range due to updating of each parameter in the parameter updating unit 32 becomes smaller than the convergence determination threshold value Δ. This is performed (step S91). Specific values of the maximum number of iterations max_iter and the threshold Δ may be, for example, max_iter = 100 and Δ = 10 ⁻¹⁰ .

In step S91, when the process in the model estimation unit 3 reaches the maximum number of iterations max_iter, or when the fluctuation range due to the update of each parameter becomes smaller than the threshold value Δ, the model estimation unit 3 determines the posterior probability γ after the end of the iteration. ^o Output _kτω .

[Second Embodiment]
The second embodiment of the present invention is an embodiment configured as a sound source number estimating device using the model estimating device A of the first embodiment.

  With reference to FIG. 3, the function structural example of the sound source number estimation apparatus B of 2nd embodiment is demonstrated. The sound source number estimation device B includes a sound source number estimation unit 4 in addition to each unit of the model estimation device A of the first embodiment.

  The sound source number estimation unit 4 estimates the number of clusters in which the posterior probability takes a significant value as the number of sound sources. Specifically, the sound source number estimation unit 4 receives the input of the posterior probability of each cluster calculated by the posterior probability calculation unit 31, and calculates the sum of the posterior probabilities of each cluster using the posterior probability of each cluster. . For example, the sound source number estimation unit 4 calculates the sum of the posterior probabilities of each cluster using the following equation (33). Note that the present invention is not limited to calculating the sum of the posterior probabilities of each cluster, and for example, a partial sum of posterior probabilities limited to a specific frequency range may be calculated.

  Then, the sound source number estimation unit 4 clusters the sum of the posterior probabilities of each cluster into two, and estimates the number of clusters belonging to the cluster with the larger sum as the number of sound sources.

  With reference to FIG. 4, the operation example of the sound source number estimation apparatus B will be described in the order of procedures. Since the processing from step S0 to step S91 is the same as the operation example of the model estimation apparatus A of the first embodiment, detailed description thereof is omitted.

  The sound source number estimation unit 4 calculates the sum of the posterior probabilities of each cluster using the posterior probabilities of each cluster (step S41). Specifically, the sound source number estimation unit 4 receives the input of the posterior probability of each cluster calculated by the posterior probability calculation unit 31, and calculates the sum of the posterior probabilities of each cluster using the posterior probability of each cluster. .

  Then, the sound source number estimation unit 4 clusters the sum of the posterior probabilities of each cluster into two (step S42). For example, the sound source number estimation unit 4 performs clustering by applying k-means clustering with two clusters to the sum of the posterior probabilities of each cluster.

  Subsequently, the sound source number estimation unit 4 estimates the number of clusters belonging to the cluster with the larger sum of the posterior probabilities as the number of sound sources (step S43). For example, the sound source number estimation unit 4 estimates the number of clusters belonging to a cluster having a larger centroid as the number of sound sources.

  Here, as exemplified above, the sound source number estimation unit 4 is not limited to clustering by applying k-means clustering with the number of clusters of “2”. Threshold processing may be performed on the sum of the posterior probabilities of each cluster using the threshold. Specifically, the sound source number estimation unit 4 determines whether the sum of the posterior probabilities of each cluster is equal to or greater than a predetermined threshold, and determines the number of clusters corresponding to the sum of the posterior probabilities equal to or greater than the predetermined threshold. May be estimated. The method of clustering by applying k-means clustering with the number of clusters of “2” is different from the method of estimating the number of clusters corresponding to the sum of the posterior probabilities equal to or higher than a predetermined threshold as the number of sound sources. Expected to be more robust against changing conditions.

  In the description of the first embodiment, it is assumed that the number of clusters exceeding the number of sound sources is set on the assumption that the number of sound sources is unknown. In this way, by setting clusters exceeding the number of sound sources and performing the parameter update processing and posterior probability calculation processing described in paragraphs 0040 to 0065, only the same number of clusters as the number of sound sources have significant posterior probabilities. The remaining clusters will have a small posterior probability. Here, FIG. 5 shows an example of the posterior probability obtained by the clustering method described in the second embodiment when the number of sound sources is “3” and the number of clusters is “4”. In FIG. 5, the horizontal axis is time, the vertical axis is frequency, and the higher the luminance, the higher the posterior probability.

  That is, it can be seen that the sound source activity has a significant value only in the same number of clusters as the actual number of sound sources. Using the example in FIG. 5, the three clusters “cluster1,” “cluster2,” and “cluster4” have significant posterior probabilities, and “cluster3” has a small posterior probability and no significant posterior probabilities. I can say that.

  For this reason, as described above, the sound source number estimation unit 4 applies the posterior of the three clusters “cluster1,” “cluster2,” and “cluster4” when clustering is performed by applying k-means clustering with two clusters. Clustering is performed using the sum of probabilities and the sum of posterior probabilities of only one cluster of “cluster3”. Then, since the number of clusters (number of elements) belonging to the cluster with the larger sum of the posterior probabilities is three, “cluster1”, “cluster2”, and “cluster4”, the number of sound sources is estimated to be “3”.

[Experimental result]
Here, an experimental result when the sound source number estimation processing by the sound source number estimation apparatus B is performed will be described.

  FIG. 6 shows the correct answer rate in the sound source number estimation experiment. As shown in FIG. 6, when φ = 600, a 100% accuracy rate was obtained under all conditions. Moreover, there was a tendency that the accuracy rate was improved by increasing φ from 1 to 600. On the other hand, when φ was further increased from 600 to 1000, the correct answer rate under the conditions of the number of sound sources N = 3 and the reverberation time 370 ms decreased from 100% to 88%.

  As described above, it is possible to appropriately estimate the number of sound sources even under conditions in which it is difficult to estimate the number of sound sources, for example, the number of sound sources> the number of microphones or reverberation. is there.

  Next, the principle of the present invention will be described. First, a sound source separation method based on clustering described in Non-Patent Document 1 will be described. The sound source separation method based on clustering described in Non-Patent Document 1 is based on the premise that the number of sound sources is known, and performs clustering using the same number of clusters as the number of sound sources. With reference to FIG. 7, the technique of said nonpatent literature 1 is demonstrated. Here, the x mark in FIG. 7 represents the sound source position feature amount (eg, phase difference, time difference, amplitude ratio between microphones) at each time frequency point. As shown in FIG. 7, the X marks of the same sound source are each surrounded by a circle, and the feature amount of one sound source tends to concentrate in the same feature amount space within each frequency. The shape of the feature amount space of each sound source depends on the frequency, but one sound source signal easily rises and falls easily at all frequencies simultaneously. That is, sound source activity is synchronized between frequencies. In such clustering, the synchronism of the sound source activity is appropriately modeled and incorporated into the overall optimization, thereby realizing simultaneous clustering of the sound source position feature quantity and the sound source activity.

  As described above, in the technique of Non-Patent Document 1, clustering is performed using the same number of clusters as the number of sound sources. Changing this to a configuration using more clusters than the number of sound sources has not been easily conceived in view of the common general technical knowledge. This is because, in the sound source separation based on clustering, if more clusters than the number of sound sources are used, normally one sound source is divided into a plurality of clusters as shown in FIG. 8, so each cluster does not correspond to a sound source. Because. FIG. 8 shows an example in which the number of sound sources is 2 and the number of clusters is 4.

  On the other hand, the following completely new knowledge was obtained in the course of research leading to the present invention. In other words, in the clustering of Non-Patent Document 1, if the configuration is changed to a configuration using more clusters than the number of sound sources, contrary to the above technical common sense, one sound source is not divided into a plurality of clusters. Group into two clusters. That is, in the clustering of the prior art, as described above, the synchronism of sound source activity is appropriately modeled, so that the situation shown in FIG. 8 does not occur, and one sound source is grouped into one cluster as shown in FIG. . FIG. 9 shows an example in which the number of sound sources is 2 and the number of clusters is 3. As shown in FIG. 9, the sound source 1 and the sound source 2 are each grouped into one cluster. The remaining one cluster has a small posterior probability.

  In the present invention, based on the above knowledge, the clustering of Non-Patent Document 1 is changed to a configuration that uses clusters larger than the number of sound sources. In the present invention, when the number of sound sources is determined based on the clustering result, the sum of the posterior probabilities of each cluster is clustered into two large and small clusters, and the number of sound sources is determined as the number of elements of the larger cluster. Thereby, compared with the case where a fixed threshold value is used, the number of sound sources can be estimated more robustly. The method of the present invention makes it possible to perform sound source separation or estimate the number of sound sources even if the number of sound sources is unknown. Further, as can be seen from the above experimental results, it was confirmed that the number of sound sources can be estimated extremely well by using the method of the present invention.

  Further, according to the sound source number estimation method of the present invention, the number of sound sources can be appropriately estimated even when “the number of sound sources> the number of microphones or reverberation”. In fact, experiments have shown that the number of sound sources can be estimated very well by the sound source number estimation method of the present invention even when “the number of sound sources> the number of microphones” or when the reverberation time is relatively long.

As described above, the sound source number estimation apparatus B uses M as a mixed signal in which signals from N sound sources are mixed, where k is a cluster index, τ is a time frame index, ω is an angular frequency. The feature vector x _τω corresponding to the observed signal vector y _τω composed of the time frequency component of each observed signal is extracted from the observed signal observed with the microphone. The sound source number estimation device B _applies the feature vector x _τω to a predetermined probability model, and uses an evaluation function that gives a higher evaluation value as the time series of the likelihood of each sound source is synchronized between frequency bins, The model parameters of the probabilistic model are estimated, and the posterior probabilities that are conditional probabilities belonging to each cluster in which the observed signal vector _yτω is set to be larger than N sound sources are calculated using the model parameters. Thereafter, the sound source number estimation device B calculates the sum of the posterior probabilities of each cluster using the posterior probability, and estimates the number of sound sources based on the sum of the posterior probabilities of each cluster. As a result, the sound source number estimation apparatus B can appropriately estimate the number of sound sources even under conditions close to the real environment.

  Further, the sound source number estimation device B clusters the sum of the posterior probabilities of each cluster into two, and estimates the number of clusters belonging to the cluster with the larger sum as the number of sound sources. The sound source number estimation apparatus B can more robustly estimate the number of sound sources than when using a threshold value.

  Further, the sound source number estimation device B determines whether the sum of the posterior probabilities of each cluster is equal to or greater than a predetermined threshold, and estimates the number of clusters corresponding to the sum of the posterior probabilities equal to or greater than the predetermined threshold as the number of sound sources. May be. In this case, the number of sound sources can be estimated more easily than in the case where clustering is performed on the above two.

[Third embodiment]
The third embodiment of the present invention is an embodiment configured as a sound source number estimation device C in which a sound source separation unit 5 and a time domain conversion unit 6 are added to the configuration of the sound source number estimation device B according to the second embodiment.

  With reference to FIG. 10, a functional configuration example of the sound source number estimation apparatus C of the third embodiment will be described. The sound source number estimation device C includes a sound source separation unit 5 and a time domain conversion unit 6 in addition to each unit of the sound source number estimation device B of the second embodiment. The sound source separation unit 5 includes a mask creation unit 51 and a separated sound creation unit 52.

  With reference to FIG. 11, an operation example of the sound source number estimation apparatus C will be described in the order of procedures. Since the processing from step S0 to step S43 is the same as the operation example of the sound source number estimation apparatus B of the second embodiment, detailed description thereof is omitted. The sound source number estimation unit 4 outputs cluster numbers k (1), k (2),..., K (^ N) for which the posterior probabilities have significant values, and is used for processing in the sound source separation unit 5. .

The sound source separation unit 5 includes a cluster number k (1), k (2),..., K (^ N) that the posterior probability output from the sound source number estimation unit 4 takes a significant value, and a frequency domain conversion unit. 1 is used to estimate the time-frequency conversion ^ s _nτω of the separated sound using the time-frequency conversion y _τω of the mixed sound output by 1 and the posterior probability γ ^o _kτω after completion of the repetition output from the posterior probability calculation unit 31 (n Is the number of the sound source.

The mask creation unit 51 includes a cluster number k (1), k (2),..., K (^ N) where the posterior probability output from the sound source number estimation unit 4 takes a significant value, and a posterior probability calculation unit. A mask m _nτω corresponding to the number of sound sources estimated by the number-of-sound sources estimation unit 4 is obtained using the posterior probability γ ^o _kτω that is output from 31 after the end of the iteration (step S51). First, the mask creation unit 51 _calculates a significant posterior probability γ ^s _nτω among posterior probabilities γ ^o _kτω after the end of the iteration, using γ ^s _nτω = γ ^o _{k (n) τω} . Here, n = 1, 2,..., ^ N, and the superscript S of γ represents significant. Next, the mask creation unit 51 uses the posterior probability γ ^s _nτω , which is a significant value after the end of the iteration, to calculate the estimated value ^ d of the active sound source number d ^s (τ, ω) by Equation (34) ^ d ^s (τ, ω) is calculated.

Next, the mask creation unit 51 calculates the mask m _nτω using the equation (35).

Note that the mask creation unit 51 may _obtain the mask m _nτω by the equation (36).

The separated sound creating unit 52 multiplies the mask m _nτω by the time frequency conversion y _1τω of the mixed sound _according to the equation (37) to calculate the time frequency conversion ^ s _nτω of the separated sound. Thereby, the observation signal in the frequency domain is separated for each sound source (step S52).

For each sound source n, the time domain conversion unit 6 converts the separation signal ^ s _nτω in the time frequency domain into a separation signal ~ ^ s _nt in the time domain and outputs it (step S6).

  As described above, the sound source number estimation device C can realize the sound source separation technique after estimating the number of sound sources even when the number of sound sources is unknown. In addition, the sound source number estimation apparatus C can realize a sound source separation technique that does not cause a permutation problem and does not require a two-stage process. Thereby, for example, it is possible to easily realize online sound source separation for speech enhancement in a time-varying environment in which the sound source position changes with time.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the model estimation unit 3 and the sound source number estimation unit 4 may be integrated. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  In addition, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
In addition, it is possible to create a program in which the processing executed by the sound source number estimation devices B and C according to the above embodiment is described in a language that can be executed by a computer. In this case, the same effect as the above-described embodiment can be obtained by the computer executing the program. Further, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer and executed to execute the same processing as in the above embodiment. Below, an example of the computer which performs the sound source number estimation program which implement | achieves the function similar to the sound source number estimation apparatuses B and C is demonstrated.

  FIG. 12 is a diagram illustrating a computer that executes a sound source number estimation program. As illustrated in FIG. 12, the computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 12, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each data described in the above embodiment is stored in, for example, the hard disk drive 1090 or the memory 1010.

  The sound source number estimation program is stored in the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described, for example. Specifically, a program module describing each process executed by the sound source number estimation apparatuses B and C described in the above embodiment is stored in the hard disk drive 1090.

  Data used for information processing by the sound source number estimation program is stored as program data in, for example, the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes the above-described procedures.

  Note that the program module 1093 and the program data 1094 related to the sound source number estimation program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and are stored in the removable storage medium by the CPU 1020 via the disk drive 1041 or the like. It may be read out. Alternatively, the program module 1093 and the program data 1094 related to the distributed data processing program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and the network interface 1070 is stored. Via the CPU 1020.

A Model estimation device B, C Sound source number estimation device 1 Frequency domain conversion unit 2 Feature extraction unit 3 Model estimation unit 31 A posteriori probability calculation unit 32 Parameter update unit 321 Mixed weight update unit 322 Correlation matrix update unit 323 Average direction update unit 324 Density Parameter updating means 325 Permutation solving means 33 Parameter holding section 4 Number of sound sources estimation section 5 Sound source separation section 51 Mask creation section 52 Separated sound creation section 6 Time domain conversion section

Claims (10)

k is the cluster index, τ is the time frame index, ω is the angular frequency,
A feature that extracts a feature vector x _τω corresponding to an observation signal vector y _τω composed of temporal frequency components of each observation signal from observation signals obtained by observing mixed signals obtained by mixing signals from N sound sources with M microphones. An extractor;
The feature vector x _τω is applied to a predetermined probability model, and the model parameters of the probability model are estimated using an evaluation function that gives a higher evaluation value as the time series of likelihood of each sound source is synchronized between frequency bins. And using the model parameter, a model estimation unit that calculates a posterior probability that is a conditional probability that the observed signal vector y _τω belongs to each cluster set more than the N sound sources;
A number-of-sound-sources estimation unit that estimates the number of clusters in which the posterior probability takes a significant value as the number of sound sources;
Including
The probability model is a mixed model represented by a weighted sum of distributions of feature vectors x _τω for each sound source,
The mixture weight of the probabilistic model is a weight that depends on the time frame τ and does not depend on the angular frequency ω,
The model parameter of the stochastic model is a parameter of the distribution of the feature weight x _τω related to each sound source and the mixture weight. The sound source number estimation apparatus according to claim 1,
The sound source number estimation unit calculates the sum of the posterior probabilities of each cluster, clusters the sum of the posterior probabilities of each cluster into two, and estimates the number of clusters belonging to the cluster with the larger sum as the number of sound sources An apparatus for estimating the number of sound sources. The sound source number estimation apparatus according to claim 1,
The sound source number estimation unit calculates the sum of the posterior probabilities for each cluster, determines whether the sum of the posterior probabilities for each cluster is equal to or greater than a predetermined threshold, and determines the sum of the posterior probabilities equal to or greater than the predetermined threshold. An apparatus for estimating the number of sound sources, wherein the number of corresponding clusters is estimated as the number of sound sources. It is a sound source number estimation apparatus as described in any one of Claim 1 to 3,
The distribution of the feature vector x _{τω with} respect to the cluster k is a Watson distribution in which the average direction is a _kω and the density parameter is κ _kω .
The number-of-sound-sources estimation device characterized in that the distribution parameter of the feature vector x _τω related to the cluster k is the average direction a _kω and the density parameter κ _kω . The sound source number estimation device according to any one of claims 1 to 4,
The sound source number estimation device according to claim 1, wherein the prior distribution of the mixture weights is a Dirichlet distribution for the mixture weights using a hyperparameter φ that does not depend on the cluster k as an index of each mixture weight. It is a sound source number estimation apparatus as described in any one of Claim 1 to 5,
The observation the model estimator is based on the product of the mixture weights at time frame τ distribution and cluster k of the feature vector x _Tauomega about the cluster k, by Moto which the feature vector x _Tauomega given corresponding to x _Tauomega A posterior probability calculator for calculating a conditional probability that the signal vector y _τω belongs to the cluster k;
A mixing weight updating means for updating the mixing weight based on the conditional probability and the hyperparameter φ independent of the cluster k;
Correlation matrix updating means for calculating a correlation matrix R _kω for cluster k based on the conditional probability and the feature vector x _τω ;
Average direction updating means for updating the average direction a _kω with the normalized principal component vector of the correlation matrix R _kω as a new value;
Density parameter updating means for updating the density parameter κ _kω based on the maximum eigenvalue of the correlation matrix R _kω ;
_Permutation solving means for reordering the average direction a _kω and the density parameter κ _kω between sound sources so that the evaluation function is maximized for each frequency bin;
A sound source number estimation device comprising: It is a sound source number estimation apparatus of Claim 6, Comprising:
The gamma _Keitauomega as the conditional probability, alpha _Lkr and the mixture weight, d (τ, ω) and contributing cluster number in the observed signal vector _y τω, the number of frequency bins F, of a · ^H · Hermitian transpose, λ _kω is the maximum eigenvalue of the correlation matrix R _kω ,
The posterior probability calculation unit calculates the conditional probability by the following equation:

The mixing weight updating means updates the mixing weight with α ′ _{kτ obtained} by the following formula as a new value,

The correlation matrix updating means updates the correlation matrix R _kW the R _'kW determined by the following equation as a new value,

The density parameter updating means updates the density parameter kappa _kW the kappa _'kW determined by the following equation as a new value

An apparatus for estimating the number of sound sources. It is a sound source number estimation apparatus as described in any one of Claim 1 to 7,
n is the number of the sound source, ^ N is the number of sound sources estimated by the sound source number estimating unit,
The sound source number estimation unit outputs cluster numbers k (1), k (2),..., K (^ N) where the posterior probability takes a significant value,
Using the cluster numbers k (1), k (2),..., K (^ N) for which the posterior probabilities have significant values, and the posterior probabilities that are conditional probabilities belonging to the respective clusters, A mask creating unit for _obtaining a mask m _nτω corresponding to the number of sound sources estimated by the sound source number estimating unit;
A separated sound creating unit that calculates a separated sound in the time-frequency domain using the mask m _nτω from the observed signal vector y _τω ;
A sound source number estimation device comprising: k is the cluster index, τ is the time frame index, ω is the angular frequency,
A step of extracting a feature vector x _τω corresponding to an observation signal vector y _τω composed of time frequency components of each observation signal from observation signals obtained by observing a mixed signal obtained by mixing signals from N sound sources with M microphones. When,
The feature vector x _τω is applied to a predetermined probability model, and the model parameters of the probability model are estimated using an evaluation function that gives a higher evaluation value as the time series of likelihood of each sound source is synchronized between frequency bins. And using the model parameters, calculating a posterior probability that is a conditional probability that the observed signal vector y _τω belongs to each cluster set more than the N sound sources;
Estimating the number of clusters for which the posterior probability takes a significant value as the number of sound sources;
Including
The probability model is a mixed model represented by a weighted sum of distributions of feature vectors x _τω for each sound source,
The mixture weight of the probabilistic model is a weight that depends on the time frame τ and does not depend on the angular frequency ω,
The model parameter of the probability model is a parameter of the number of sound sources, characterized in that the mixture weight and a parameter of the distribution of the feature vector x _τω for each sound source.   A sound source number estimation program for causing a computer to function as the sound source number estimation device according to claim 1.

Images