JP2016156944A - Model estimation device, target sound enhancement device, model estimation method, and model estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve sound source separation with higher precision even when a reverberation time is long as compared with a frame length.SOLUTION: A model estimation device 10A comprises a storage part which stores a parameter of a model for a mixture signal, including reverberation, including a regression matrix showing characteristics of reverberation due to sounds output from a plurality of sound sources. The model estimation device 10A estimates a mixture signal including no reverberation through linear prediction using an observation signal obtained by observing a sound by a plurality of microphones and a regression matrix stored in a storage part. The model estimation device 10A calculates a posterior probability corresponding to a sound source that each time frequency point belongs to from the estimated mixture signal. The model estimation device 10A estimates a parameter from the observation signal, the estimated mixture signal, posterior probability, and parameter stored in the storage part, and updates the parameter stored in the storage part with the estimated parameter. The model estimation device 10A repeats the signal estimation, clustering and parameter estimation until a predetermined condition is met.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a model estimation device, a target sound enhancement device, a model estimation method, and a model estimation program.

  Conventionally, there is a sound source separation technique as a target sound enhancement technique. The sound source separation technique is a technique for estimating each sound source signal using a mixed signal of a plurality of sound source signals acquired by a plurality of microphones. In particular, a sound source separation technique based on clustering and a sound source separation technique based on independent component analysis are well known. Hereinafter, as a conventional technique, a sound source separation technique based on clustering will be described. In the following, for example, when A is a vector, it is expressed as “vector A”, and when A is a scalar, for example, it is simply expressed as “A”. In the following description, signal expression in the time frequency domain is used unless otherwise specified. The time frame number is represented by t∈ {1, 2,..., T} (T is the total number of frames), and the frequency bin number is represented by f∈ {1, 2,. The total number of frequency bins below the Nyquist frequency).

  The signal representation in the time-frequency domain can be obtained by applying a time-frequency transform such as a short-time Fourier transform to the signal representation in the time domain. Conversely, the signal representation in the time domain can be obtained by applying the inverse transform of the time frequency transform such as the inverse short time Fourier transform to the signal representation in the time frequency domain.

Assume that signals from N sound sources (N is a natural number) are observed with M (M is a natural number) microphones. The mixed signal observed by the m (1 ≦ m ≦ M) -th microphone is represented by y ^(m) _tf , and the mixed signal observed by the M microphones is expressed by the mixed signal vector y as shown in the following equation (1). _Expressed collectively as _tf .

In the above formula (1), · ^T represents transposition of ·. When the reverberation time is sufficiently shorter than the frame length, the mixed signal vector y _tf can be modeled by the following equation (2).

In the above equation (2), c ⁽ⁿ⁾ _tf represents the nth sound source signal. Further, the vector h ⁽ⁿ⁾ _f in the above equation (2) is defined by the following equation (3). In the following equation (3), h ^{(m, n)} _f represents a time-invariant transfer function from the nth sound source signal to the mth microphone.

The vector h ⁽ⁿ⁾ _f is called a steering vector and includes information on the position of the nth sound source. In the following, for simplicity, it is assumed that the number of microphones is M = 2, the influence of reverberation and reverberation can be ignored, and each sound source signal propagates as a plane wave. In this case, the vector h ⁽ⁿ⁾ _f can be modeled by the following equation (4). In the following formula (4), “j” represents an imaginary unit.

Here, ω _f in the above equation (4) represents the angular frequency corresponding to the number f of the frequency bin, d ^{(m, n)} represents the distance between the m-th microphone and the n-th sound source, and c Represents the speed of sound. An arrival time difference δ ⁽ⁿ⁾ between microphones of the n-th sound source is defined by the following equation (5).

Then, the phase difference between microphones in the steering vector h ⁽ⁿ⁾ _f arg (h ^{(1, n)} _f ) −arg (h ^{(2, n)} _f ) (arg (•) is the declination (phase) of And the nth inter-microphone arrival time difference δ ⁽ⁿ⁾ has a relationship represented by the following equation (6).

In the sound source separation technique based on clustering, the observed mixed signal vector y _tf is assumed to be “consisting of only a single sound source component at each time frequency point” (hereinafter referred to as “sparse”) (for example, non Patent Document 1). It is known that sparse is accurately established when the influence of reverberation is small and the sound source signal is speech. Under the sparse assumption, the number of the sound source “included in the mixed signal vector y _tf at the time frequency point (t, f)” (hereinafter referred to as “active”) is represented by d _tf , and the above (2) The equation can be rewritten as the following equation (7).

Under the assumption of sparsity, the feature quantity z _tf calculated from the observed signal based on the definition of the following equation (8) is the distance between microphones of the active d _tf sound source as shown in the following equation (9): It matches the arrival time difference.

Therefore, sound source separation can be realized by clustering z _tf . Clustering can be performed, for example, by clustering techniques such as mixed model fitting and k-means clustering (see, for example, Non-Patent Document 2).

O. Yilmaz and S. Rickard, “Blind separation of speech mixture via time-frequency masking.” IEEE Trans. SP, vol. 52, no. 7, pp. 1830-1847, Jul. 2004. S. Araki, H. Sawada, R. Mukai, and S. Makino, “Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors.” Signal Processing, vol. 87, no. 8, pp. 1833-1847, Aug. 2007 . Nobutaka Ito, Akiko Araki, Keisuke Kinoshita, Tomohiro Nakatani, “Permutation Free Clustering Method for Underdetermined Blind Source Separation Based on Source Location Information”, IEICE Transactions, vol. J97-A, no. 4 , pp. 234-246, Apr. 2014. T. Yoshioka, T. Nakatani, M. Miyoshi, and HG Okuno, “Blind separation and dereverberation of speech mixture by joint optimization.” IEEE Trans. ASLP, vol. 19, no. 1, pp. 69-84, Jan. 2011. NQK Duong, E. Vincent, and R. Gribonval, “Under-determined reverberant audio source separation using a full-rank spatial covariance model.” IEEE Trans. ASLP, vol. 18, no. 7, pp. 1830-1840, Sep . 2010. AP Dempster, NM Laird, and DB Rubin, “Maximum likelihood from incomplete data via the EM algorithm.” Journal of the Royal Statistical Society: Series B (Methodological), vol. 39, no. 1, pp. 1-38, 1977 . H. Sawada, S. Araki, and S. Makino, “Underdetermined convolutive blind source separation via frequency bin-wise clustering and permutation alignment.” IEEE Trans. ASLP, vol. 19, no. 3, pp. 516-527, Mar . 2011.

  However, since the above prior art is based on the premise that the reverberation time is sufficiently short compared with the frame length, there is a problem that the sound source separation performance is deteriorated in many real environments (for example, conference rooms) where this premise is not satisfied. is there.

  An example of the embodiment disclosed in the present application has been made in view of the above, and an object thereof is to realize more accurate sound source separation even when the reverberation time is longer than the frame length.

  In an example of the embodiment of the present application, the model estimation apparatus includes a storage unit that stores a model parameter of a mixed signal including reverberation including a regression matrix indicating characteristics of reverberation due to sounds output from a plurality of sound sources. The model estimation apparatus estimates a mixed signal that does not include reverberation by linear prediction using observation signals obtained by observing sound with a plurality of microphones and a regression matrix stored in a storage unit. The model estimation apparatus clusters the estimated mixed signal into clusters for each sound source to which each time frequency point belongs, and calculates a posteriori probability corresponding to each cluster from parameters stored in the storage unit. The model estimation apparatus estimates a parameter from the estimated mixed signal and the calculated posterior probability, and updates the parameter stored in the storage unit with the estimated parameter. The model estimation apparatus repeats signal estimation, clustering, and parameter estimation until a predetermined condition is satisfied.

  According to an example of the embodiment disclosed in the present application, for example, even when the reverberation time is longer than the frame length, more accurate sound source separation can be realized.

FIG. 1 is a diagram illustrating an example of a configuration of a model estimation apparatus according to the first embodiment. FIG. 2 is a flowchart illustrating an example of a processing procedure of the model estimation apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of the configuration of the model estimation apparatus according to the second embodiment. FIG. 4 is a diagram illustrating an example of the configuration of the model estimation apparatus according to the third embodiment. FIG. 5 is a flowchart illustrating an example of a processing procedure of the model estimation apparatus according to the third embodiment. FIG. 6 is a diagram illustrating an example of the configuration of the target sound enhancement device according to the fourth embodiment. FIG. 7 is a flowchart illustrating an example of a processing procedure of the target sound enhancement device according to the fourth embodiment. FIG. 8 is a diagram for explaining an example of the effect of the fourth embodiment. FIG. 9 is a diagram illustrating an example of the effect of the fourth embodiment. FIG. 10 is a diagram illustrating an example of a computer in which a model estimation device and a target sound enhancement device are realized by executing a program.

[Embodiment]
Hereinafter, embodiments of a model estimation device, a target sound enhancement device, a model estimation method, and a model estimation program disclosed in the present application will be described. The following embodiments are merely examples, and do not limit the technology disclosed by the present application. Moreover, you may combine suitably each embodiment shown below in the range which does not contradict.

In the following embodiments, for example, when A is a vector, it is expressed as “vector A”, and when A is a scalar, for example, it is simply expressed as “A”. For example, when A is a set, it is described as “set A”. For example, the function f of the vector A is expressed as f (vector A). Further, with respect to A is a vector or scalar, be referred to as a "~ ^A" is assumed to be equivalent to "" A "symbol" ~ "is written directly above the". Further, when “^ A” is described for A which is a vector or a scalar, it is equivalent to “a symbol in which“ ^ ”is written immediately above“ A ””. In addition, when “˜ ^ A” is written for A which is a vector or a scalar, it is equivalent to “a symbol where“ ^ ”is written immediately above“ A ”and“ ˜ ”is added immediately above” ”. Suppose that In addition, A ^T represents transposition of A with respect to A which is a vector or a scalar. Further, with respect to the matrix A, the matrix A- ¹ represents the inverse matrix of the matrix A, detA represents the determinant of the matrix A, and trA represents the diagonal sum (trace) of the matrix A. Further, with respect to the matrix A, the matrix A ^H represents the Hermitian transpose of the matrix A, and the matrix A ^* represents the complex conjugate of the matrix A. For set A, #A represents the number of elements in set A. Exp (·) is an exponential function, and ln (·) is a logarithmic function.

[Embodiment 1]
Hereinafter, after describing the theoretical background of the first embodiment, one aspect of the first embodiment will be described.

<Theoretical Background of Embodiment 1>
In the first embodiment, it is assumed that signals from N (N is a natural number) sound sources are observed with M (M is a natural number) microphones under reverberation. The mixed signal including reverberation observed by the m (1 ≦ m ≦ M) -th microphone is represented by y ^(m) _tf , and the mixed signal observed by M microphones is mixed as shown in the following equation (10). _These are collectively expressed as a signal vector y _tf .

The model estimation apparatus according to the first embodiment applies a mixed signal vector y _tf including reverberation to a probability model representing the distribution of the mixed signal vector y _tf and estimates a parameter of a predetermined probability model. Hereinafter, first, a probability model representing the distribution of the mixed signal vector y _tf will be described, and then an algorithm for estimating the parameters of the probability model representing the distribution of the mixed signal vector y _tf will be derived. Hereinafter, the theoretical background will be described for each of modeling of the mixed signal vector including reverberation and derivation of the parameter estimation algorithm.

(Modeling of mixed signal vector including reverberation in embodiment 1)
When it is assumed that only the nth (1 ≦ n ≦ N) sound source exists and no reverberation and other sound sources exist, a vector (hereinafter referred to as “n”) in which signals scheduled to be observed by M microphones are arranged. ^(The microphone image that does not include the reverberation of the second sound source) is represented by a vector s ⁽ⁿ⁾ _tf ∈ set C ^M. Here, the vector s ⁽ⁿ⁾ _tf is an M-dimensional vector whose elements are complex numbers. Assuming that no reverberation exists, a vector in which mixed signals to be observed by M microphones are arranged (hereinafter referred to as a “mixed signal vector not including reverberation”) is expressed as x _tf ∈ set C ^M Represent. Assuming a mixed signal vector x _tf free of reverberation is sparse, mixed signal vector x _tf can be modeled by the following equation (11).

In the sound source separation based on the conventional clustering, the mixed signal vector y _tf including reverberation is assumed to be sparse, whereas the first embodiment assumes that the mixed signal vector x _tf not including reverberation is sparse. This enables accurate modeling even under reverberation. Based on the model of the mixed signal vector x _tf according to the above equation (11), the distribution of the mixed signal vector x _tf not including reverberation is modeled by the mixture distribution of the following equation (12).

In the above equation (12), p (vector s ⁽ⁿ⁾ _tf | Θ) represents a probability model representing the distribution of the vector s ⁽ⁿ⁾ _tf of the microphone image that does not include the reverberation of the nth sound source. In the above equation (12), P (d _tf | Θ) is called a mixing weight and represents a probability model of the active sound source number d _tf . In the above equation (12), Θ represents a set of parameters of the probability model. The definition of the set Θ will be described later.

On the other hand, the mixed signal vector y _tf including reverberation can be modeled by the following equation (13) by a multichannel autoregressive process driven by the mixed signal vector x _tf not including reverberation. For modeling of the mixed signal vector y _tf , reference 1 “T. Yoshioka, T. Nakatani, M. Miyoshi, and HG Okuno,“ Blind separation and dereverberation of speech mixture by joint optimization. ”IEEE Trans. ASLP, vol. 19, no. 1, pp. 69.84, Jan. 2011. ”.

Here, in the above equation (13), k represents a tap number, K represents the number of taps, a matrix G _kf ∈ set C ^{M × M} represents an M-row M-column regression matrix having complex numbers as elements, The matrix G ^H _kf represents the Hermitian transpose of the regression matrix G _kf . In the above equation (13), Δ represents a predetermined delay, but is preferably set so as to correspond to a time during which the sound source signal has autocorrelation (in the case of speech, about 20 to 30 ms). By introducing the delay Δ, the autocorrelation of the sound source signal is prevented from being removed when dereverberation is performed using the estimated regression matrix G _kf . For convenience, the mixed signal vector y _tf = 0 (zero vector) is defined for t <0. For convenience, when the model of the above equation (13) is expressed as a probability model, the following equation (14) is obtained. In the following equation (14), δ is a Dirac delta function.

Using the probability models of the above equations (12) and (14), a probability model representing the distribution of the mixed signal vector y _tf including reverberation can be derived as in the following equations (15) and (16). .

In the derivation of the probability model of the above equation (16) representing the distribution of the mixed signal vector y _tf including reverberation, a probability model p (representing the distribution of the vector s ⁽ⁿ⁾ _tf of the microphone image not including the reverberation of each sound source. Note that no assumptions are made about the concrete form of the vector s ⁽ⁿ⁾ _tf | Θ) and the probability model P (d _tf | Θ) of the active sound source number d _tf . That is, even if these probability models are modeled by an arbitrary probability distribution, a probability model representing the distribution of the mixed signal vector y _tf including reverberation is given by the above equation (16).

According to the above equation (16), the probability model representing the distribution of the mixed signal vector y _tf including reverberation is determined by the probability model P (d _tf | Θ) of the active sound source number d _tf and each sound source. It can be seen that this results in defining a probability model p (vector s ⁽ⁿ⁾ _tf | Θ) representing the distribution of the vector s ⁽ⁿ⁾ _tf of the microphone image that does not include reverberation. Although these probability models can be modeled using arbitrary probability distributions, these modeling in the first embodiment will be described below.

The probability model p (vector s ⁽ⁿ⁾ _tf | Θ) representing the distribution of the vector s ⁽ⁿ⁾ _tf of the microphone image not including the reverberation of the nth sound source is, for example, a time-varying Gaussian distribution of the following equation (17). Can be modeled. For this modeling, refer to Reference 2, “NQK Duong, E. Vincent, and R. Gribonval,“ Under-determined reverberant audio source separation using a full-rank spatial covariance model. ”IEEE Trans. ASLP, vol. 18, no. 7, pp. 1830.1840, Sep. 2010. ”.

Here, in the above equation (17), φ ⁽ⁿ⁾ _tf is a parameter for modeling the time-varying power spectrum of the vector s ⁽ⁿ⁾ _tf , and the matrix B ⁽ⁿ⁾ _f is the vector s ⁽ⁿ⁾ It is a parameter that models the time-invariant spatial covariance matrix of _tf . The right side of the above equation (17) is a probability density function of a complex Gaussian distribution expressed by the following equation (18). The following equation (18) represents a probability density function of a complex Gaussian distribution in which a random variable is a vector α, an average is a vector μ, and a covariance matrix Σ. In the following equation (18), π represents a circular ratio, and det (πΣ) represents a determinant of a matrix πΣ.

In the first embodiment, the probability model P (d _tf | Θ) of the active sound source number d _tf is modeled by the following equation (19) using the frequency-dependent mixture weight α ⁽ⁿ⁾ _f .

The concrete form of the probability model representing the distribution of the mixed signal vector y _tf including reverberation in the first embodiment is a microphone image vector s that does not include the reverberation of the nth sound source in the above-described equation (16), which is a general case. ⁽ⁿ⁾ the probability model representing the distribution of _tf p (vector s ⁽ⁿ⁾ _tf | theta) and the formula (17) is a specific form of the probability model P number d _tf active sound sources (d _tf | theta) By substituting the above formula (19) which is a specific form of the following formula, the following formula (20) is obtained.

  Here, the parameter set Θ is specifically defined by the following equation (21).

(Derivation of Parameter Estimation Algorithm of Embodiment 1)
Based on the above equation (16) indicating the probability model of the mixed signal vector y _tf including reverberation, the set of parameters Θ can be estimated, for example, according to the maximum likelihood method or the MAP (Maximum A Posteriori) estimation method.

In the maximum likelihood method, the likelihood p (Y | Θ) of the mixed signal vector y _tf including reverberation is used as an evaluation function, and the likelihood p (Y | Θ) is maximized to estimate the parameter set Θ = arg Find max _Θ {p (Y | Θ)}. Here, the set Y is defined as Y: = {vector y _tf } _tf : = {vector y _tf | ∀t, f}.

On the other hand, in the MAP estimation method, the posterior probability p (Θ | Y) of the parameter set Θ is used as an evaluation function, and the estimated value Θ = arg max _Θ of the parameter set is maximized by maximizing the posterior probability p (Θ | Y). {P (Θ | Y)} is obtained. Furthermore, MAP is noted from the Bayes' theorem that p (Θ | Y) = {p (Y | Θ) p (Θ)} / p (Y) and p (Y) is a constant. The estimated value of the parameter set Θ by the estimation method can be rewritten as the following equation (22). In the following equation (22), p (Θ) represents the prior probability of the parameter set Θ.

The likelihood p mixed signal vector y _tf containing reverberation (Y | theta), using a probability model representing the distribution of the mixed signal vector y _tf appear on the left-hand side of the equation (15), including reverberation, following (23 ) Expression.

  The prior probability p (Θ) of the parameter set Θ can be modeled using an arbitrary probability model, but a uniform distribution can be used, for example. When the uniform distribution is used, the estimated value of the parameter set Θ by the MAP estimation method based on the above equation (22) coincides with the maximum likelihood estimation. Alternatively, a Dirichlet distribution such as the following equation (24) is assumed as the prior distribution of the mixture weight.

  A uniform prior distribution may be assumed for parameters other than the mixture weight. In this case, the prior distribution P (Θ) of the set of parameters Θ is proportional to the prior distribution of the mixture weights shown in the above equation (24). Here, ψ in the above equation (24) is a predetermined constant called a hyper parameter. ψ can be set to an arbitrary positive number. For example, ψ = 600 may be set.

In the following, as an example of an algorithm for estimating the parameter set Θ by the MAP estimation method based on the above equation (22), an EM (Expectation-Maximization) algorithm that considers the set D: = {d _tf } as a hidden variable Is derived. Regarding EM, Reference 3 “AP Dempster, NM Laird, and DB Rubin,“ Maximum likelihood from incomplete data via the EM algorithm. ”Journal of the Royal Statistical Society: Series B (Methodological), vol. 39, no. 1, pp. 1.38, 1977 ”.

  The EM algorithm is to repeat the E step and M step defined below until the convergence condition is satisfied. In the E step, a Q function defined by the following equation (25): Q (Θ; Θ ′) is calculated.

Where lnP (Y, D | Θ) represents the log likelihood of the complete data set {Y, D}, and P (D | Y, Θ ′) is the current estimate Θ of the parameter set Θ. Represents the posterior probability of the set D with respect to ′, and </ _{P (D | Y, Θ ′)} represents the expected value calculation for P (D | Y, Θ ′).

  On the other hand, in the M step, the parameter set Θ is updated by maximizing the Q function. In each iteration of the EM algorithm, the monotonic non-decreasing property of the evaluation function p (Y | Θ) p (Θ) is guaranteed. In order to calculate the specific form of the Q function, p (Y, D | Θ) and p (D | Y, Θ ′) are obtained as in the following equations (26) to (30).

Here, γ ⁽ⁿ⁾ _tf in the above equation (30) is defined by the following equations (31) to (33).

However, for the sake of simplicity, in the above equation (33), the current estimated values of α ⁽ⁿ⁾ _f , regression matrix G _kf , φ ⁽ⁿ⁾ _tf , and matrix B ⁽ⁿ⁾ _f are simply α ⁽ⁿ⁾ , respectively. _f , regression matrix G _kf , φ ⁽ⁿ⁾ _tf , matrix B ⁽ⁿ⁾ _f .

  By substituting the above equations (28) and (30) into the above equation (25), specific forms of the Q function can be obtained as the following equations (34) and (35).

The update formula of the mixture weight α ⁽ⁿ⁾ _f is obtained by using the Lagrange's undetermined multiplier method while paying attention to the constraint condition Σ ^N _{n = 1} α ⁽ⁿ⁾ _f = 1. phi ⁽ⁿ⁾ _tf and update equation of the matrix B ⁽ⁿ⁾ _f is the ⁽³⁵⁾ φ ⁽ⁿ⁾ _tf of Q function shown in the expression and the complex conjugate of the matrix B ⁽ⁿ⁾ _f (matrix B ^{( n)} It is obtained by setting the partial derivative with respect to _f ) ^* to 0.

Update equation of the regression matrix G _kf from Q function shown in the expression (35), when extracting only the term that depends on a Hermitian transpose of regression matrix G _kf matrix G ^H _kf, following equation (36), (37 )

If the partial differentiation with respect to the matrix to G ^H _{f in the} equation (37) is set to 0, the following equation (41) is obtained.

  By applying the vec operator to both sides of the above equation (41) and applying the formula for the Kronecker product for the matrix A, the matrix B, and the matrix X shown by the following equation (a), the following equation (42) is obtained. become.

However, in the above equation (42), vec [a ₁ ... A _P ] and matrices to G ^H _f are defined as the following equations (43) and (44), respectively.

Therefore, from the above equation (42), vec [matrix˜G ^H _f ] is obtained as in the following equation (45).

<One aspect of Embodiment 1>
Hereinafter, one aspect of the first embodiment based on the theoretical background of the first embodiment will be described. In one aspect of Embodiment 1, it is assumed that the number of sound sources N is known.

(Configuration of Model Estimation Device According to Embodiment 1)
FIG. 1 is a diagram illustrating an example of a configuration of a model estimation apparatus according to the first embodiment. The model estimation apparatus 10A according to the first embodiment includes a dereverberation processing unit 11A and a clustering unit 12A. The dereverberation processing unit 11A includes an initialization unit 11A-1, a covariance matrix update unit 11A-2, a regression matrix update unit 11A-3, and a dereverberation unit 11A-4. The covariance matrix update unit 11A-2, the regression matrix update unit 11A-3, and the mixture weight update unit 12A-2 are examples of a parameter estimation unit. The dereverberation unit 11A-4 is an example of a signal estimation unit. The posterior probability update unit 12A-1 is an example of a posterior probability calculation unit.

The initialization unit 11A-1 first calculates an initial value of the parameter set Θ. This initial value can be calculated as follows, for example. First, an estimate ^ d _tf number d _tf active sound source is calculated using the sound source separation technique based on the conventional clustering without the reverberation model. The conventional sound source separation technique based on clustering without reverberation model is described in Reference 4 “Nobutaka Ito, Akiko Araki, Keisuke Kinoshita, Tomohiro Nakatani,“ Permutation-free clustering method for underdetermined blind sound source separation based on sound source position information. ", IEICE Transactions, vol. J97-A, no. 4, pp. 234.246, Apr. 2014."

Next, the initialization unit 11A-1 uses the estimated value ^ d _tf to initialize each parameter according to the following formulas (46) to (49). The set C ⁽ⁿ⁾ _f in the following formulas (46) and (48) is a matrix defined by C ⁽ⁿ⁾ _f : = {t | d _tf = n}. Also, #C ⁽ⁿ⁾ _f in the following formulas (46) and (48) represents the number of elements of the set C ⁽ⁿ⁾ _f . Also, “tr [•]” in the following equation (49) represents a trace of the matrix [•].

The covariance matrix updating unit 11A-2 uses the covariance matrix φ ⁽ⁿ⁾ _tf B ^{(n )} of the vector s ⁽ⁿ⁾ _tf of the microphone image not including the reverberation of each sound source n (n = 1,..., N). ⁾ parameters phi ⁽ⁿ⁾ _tf and matrix B ⁽ⁿ⁾ _f of _f, respectively following equation (50), updated by (51) below.

The regression matrix updating unit 11A-3 updates the regression matrix G _kf with the following formulas (52) and (53).

Here, the matrix ~ G _f appearing on the left side of the above expression (53) and the vector ~ y _{t-Δ-1, f} appearing on the right side of the above expression (53) are expressed by the following expressions (54) and (55). Defined in

The dereverberation unit 11A-4 updates the estimated value _xtf of the mixed signal vector not including reverberation according to the following equation (56).

The clustering unit 12A includes a posterior probability update unit 12A-1 and a mixture weight update unit 12A-2. The posterior probability update unit 12A-1 sets the posterior probability γ ⁽ⁿ⁾ _tf that the n (n = 1,..., N) th sound source signal is active at the time frequency point (t, f) as follows (57 Update with the formula. Note that γ ⁽ⁿ⁾ _tf : = P (d _tf = n | vector y _tf , Θ).

The mixing weight updating unit 12A-2 updates the mixing weight α ⁽ⁿ⁾ _f by the following equation (58).

In addition, in order to improve performance, whitening as described below may be performed as preprocessing on the mixed signal vector y _tf including reverberation prior to the entire processing of the model estimation apparatus 10A.

In the first embodiment, the a posteriori probability updating unit 12A-1 of the clustering unit 12A performs n (n = 1,..., N) th time frequency points (t, f) based on the above equation (57). Suppose we calculate the posterior probability γ ⁽ⁿ⁾ _tf that the sound source signal is active. However, the present invention is not limited to this, and a conventional technique such as k-means clustering is used, and the posterior probability γ that the n (n = 1,..., N) th sound source signal is active at the time frequency point (t, f). ⁽ⁿ⁾ _tf may be calculated.

(Processing of model estimation apparatus according to Embodiment 1)
FIG. 2 is a flowchart illustrating an example of a processing procedure of the model estimation apparatus according to the first embodiment. The process of the model estimation apparatus 10A described below is repeated until a predetermined convergence determination condition is satisfied. The predetermined convergence condition is, for example, “A predetermined number of iterations has been reached, or before and after update by one or more update units among the update units of the posterior probability update unit 12A-1 and the mixed weight update unit 12A-2. The difference between the parameter values is less than a predetermined threshold value ”.

First, in step S11, the initialization unit 11A-1 calculates the initial value of the parameter set Θ based on the equations (46) to (49), and stores it in the main storage device of the model estimation device 10A. Next, in step S12, the dereverberation unit 11A-4, based on the regression matrix G _kf currently stored in the main storage device of the model estimation apparatus 10A, _uses the above equation (56) to _{obtain a} mixed signal vector that does not include reverberation. Update the estimated value ^ x _tf of “(reverberation removal)”.

Next, in step S13, the posterior probability update unit 12A-1 is the time-frequency point (t, f) by n (n = 1, ···, N) th posterior probability of the sound source signal is active gamma ^{(n )} _tf is calculated by the above equation (57) and stored in the main memory of the model estimation apparatus 10A. In step S13, the mixture weight updating unit 12A-2 calculates the mixture weight α ⁽ⁿ⁾ _f by the above equation (58) and stores it in the main storage device of the model estimation apparatus 10A (hereinafter, “clustering”). processing).

  Next, the model estimation apparatus 10A determines whether or not the convergence determination condition is satisfied (step S14). When the convergence determination condition is satisfied (Yes in step S14), the model estimation device 10A ends the process. If the convergence determination condition is not satisfied (No at Step S14), the model estimating apparatus 10A moves the process to Step S15.

In step S15, the covariance matrix updating unit 11A-2 performs the covariance matrix φ ^{(n) of the} microphone image vector s ⁽ⁿ⁾ _tf that does not include the reverberation of each sound source n (n = 1,..., N ^). the _tf B ⁽ⁿ⁾ parameters _f phi ⁽ⁿ⁾ _tf and matrix B ⁽ⁿ⁾ _f, each of the above equation (50), calculated by (51) below, updates stored in the main memory of the model estimation device 10A. In step S15, the regression matrix update unit 11A-3 calculates the regression matrix G _kf based on the parameters φ ⁽ⁿ⁾ _tf and matrix B ⁽ⁿ⁾ _f calculated by the covariance matrix update unit 11A-2. Calculations are made according to equations (52) and (53), and updated and stored in the main storage device of the model estimation device 10A.

In step S15, the posterior probability update unit 12A-1 also includes a set of parameters Θ currently stored in the main storage device of the model estimation apparatus 10A, and a mixed signal vector that does not include the reverberation in step S12 that was executed last. Posterior probability γ ⁽ⁿ⁾ _tf is calculated by the above equation (57) based on the estimated value ^ x _tf and updated and stored in the main memory of the model estimating apparatus 10A. In step S15, the mixture weight update unit 12A-2 uses the above equation (58) to calculate the mixture weight α ⁽ⁿ⁾ _f based on the posterior probability γ ⁽ⁿ⁾ _tf calculated by the posterior probability update unit 12A-1. Are updated and stored in the main storage of the model estimation apparatus 10A. When the process in step S15 is completed, the model estimation device 10A moves the process to step S12.

[Embodiment 2]
Hereinafter, after describing the theoretical background of the second embodiment, one aspect of the second embodiment will be described.

<Theoretical Background of Embodiment 2>
As in the first embodiment, when the frequency-dependent mixture weight shown in the above equation (19) is used, the posterior probability that is the evaluation function has indefiniteness of permutation (replacement). That is, the order of α ⁽ⁿ⁾ _f , φ ⁽ⁿ⁾ _tf , and matrix B ⁽ⁿ⁾ _tf of the parameter set Θ is expressed by the following equation (62) by the permutation Π _f on {1 _,. Consider the case of replacement.

  At this time, the following equation (63) holds.

That is, there is a permutation problem that the number n in the estimated Θ corresponds to a different sound source for each frequency only by maximizing the posterior probability. Therefore, the target sound cannot be properly emphasized by using the estimated Θ as it is. Therefore, when the target sound enhancement device is configured based on the first embodiment, a permutation resolution process is separately required for determining the replacement Π _f so that the number n corresponds to the same sound source regardless of the frequency. It becomes.

  In contrast, the model estimation apparatus according to the second embodiment uses time-dependent mixture weights. Thereby, as disclosed in the above-mentioned document 4, model estimation is possible without causing a permutation problem by maximizing the posterior probability.

  Hereinafter, the theoretical background of the second embodiment will be described with an emphasis on the difference from the first embodiment.

(Modeling of mixed signal vector including reverberation in embodiment 2)
In the second embodiment, the probability model P (d _tf | Θ) of the active sound source number d _tf is _expressed by the following (64) using the time-dependent mixture weight α ⁽ⁿ⁾ _t instead of the frequency-dependent mixture weight. Model with an expression.

Therefore, the concrete form of the probability model (see the above equation (16)) representing the distribution of the mixed signal vector y _tf including reverberation in the second embodiment is obtained as the following equation (65).

  The parameter set Θ is specifically expressed by the following equation (66).

(Derivation of Parameter Estimation Algorithm of Embodiment 2)
The second embodiment is the same as the first embodiment in that the EM algorithm maximizes the posterior probability. However, in the second embodiment, in each iteration of the EM algorithm, a P (Permutation) step process is performed in addition to the E step and M step processes. In the P step, permutation is performed by replacing the covariance matrix φ ⁽ⁿ⁾ _tf B ⁽ⁿ⁾ _f between sound sources so that the posterior probability that is the objective function is maximized at the number f of each frequency bin. Solve the problem. That is, Π _f is substituted on {1,..., N}, and the following formulas (67) to (69) are processed.

  Note that the derivation of the update formula in the E step and the M step is the same as in the first embodiment, and thus the description thereof is omitted.

<One aspect of Embodiment 2>
Hereinafter, an aspect of the second embodiment based on the theoretical background of the second embodiment will be described. In one aspect of the second embodiment, the number N of sound sources is assumed to be known. However, the second embodiment assumes that the upper limit is known even if the true sound source number N ₀ is not known, and sets the assumed sound source number N to be larger than the upper limit of the true sound source number N _0. Thus, the present invention can be implemented in the same manner as when the number of sound sources is known.

(Configuration of model estimation apparatus according to Embodiment 2)
FIG. 3 is a diagram illustrating an example of the configuration of the model estimation apparatus according to the second embodiment. The model estimation apparatus 10B according to the second embodiment includes a dereverberation processing unit 11B and a clustering unit 12B. The dereverberation processing unit 11B includes an initialization unit 11B-1, a covariance matrix update unit 11B-2, a regression matrix update unit 11B-3, and a dereverberation unit 11B-4. The covariance matrix update unit 11B-2, the regression matrix update unit 11B-3, and the mixture weight update unit 12B-2 are examples of a parameter estimation unit. The dereverberation unit 11B-4 is an example of a signal estimation unit. The posterior probability update unit 12B-1 is an example of a posterior probability calculation unit.

The initialization unit 11B-1 first calculates an initial value of the parameter set Θ. This initial value can be calculated as follows, for example. First, an estimate ^ d _tf number d _tf active sound sources, as in the first embodiment, calculated using the sound source separation technique based on the conventional clustering without the reverberation model. Next, the initialization unit 11B-1 initializes each parameter using the estimated value ^ d _{tf according} to the above equations (47) to (49) and the following equation (70). Note that the set in the following equation (70) to C ⁽ⁿ⁾ _t is a matrix defined by C ⁽ⁿ⁾ _t : = {f | d _tf = n}. Also, #C ⁽ⁿ⁾ _t in the following equation (70) represents the number of elements of the set C ⁽ⁿ⁾ _t .

  The covariance matrix update unit 11B-2, the regression matrix update unit 11B-3, and the dereverberation unit 11B-4 are the covariance matrix update unit 11A-2, the regression matrix update unit 11A-3, and the dereverberation unit 11A of the first embodiment. Same as -4.

The clustering unit 12B includes a posterior probability update unit 12B-1, a mixture weight update unit 12B-2, and a permutation resolution unit 12B-3. The posterior probability updating unit 12B-1 sets the posterior probability γ ⁽ⁿ⁾ _tf that the n (n = 1,..., N) th sound source signal is active at the time frequency point (t, f) as follows (71 Update with the formula. Note that γ ⁽ⁿ⁾ _tf : = P (d _tf = n | vector y _tf , Θ).

The mixing weight updating unit 12B-2 updates the mixing weight α ⁽ⁿ⁾ _t by the following equation (72).

The permutation resolution unit 12B-3 replaces the covariance matrix φ ⁽ⁿ⁾ _tf B ⁽ⁿ⁾ _f between sound sources so that the posterior probability that is an objective function is maximized at the number f of each frequency bin. To solve permutation. That is, Π _f is substituted on {1,..., N}, and the covariance matrix φ ⁽ⁿ⁾ _tf B ⁽ⁿ⁾ _f is substituted by the following equations (73) to (75).

In order to improve the performance, whitening shown in the above equations (59) to (61) is performed as preprocessing on the mixed signal vector y _tf including reverberation prior to the entire processing of the model estimation apparatus 10B. Good.

In the second embodiment, the a posteriori probability updating unit 12B-1 of the clustering unit 12B performs n (n = 1,..., N) th time frequency points (t, f) based on the above equation (71). Suppose we calculate the posterior probability γ ⁽ⁿ⁾ _tf that the sound source signal is active. However, the present invention is not limited to this, and a conventional technique such as k-means clustering is used, and the posterior probability γ that the n (n = 1,..., N) th sound source signal is active at the time frequency point (t, f). ⁽ⁿ⁾ _tf may be calculated.

[Embodiment 3]
In the third embodiment, the number of sound sources is also estimated using the model estimation device 10B of the second embodiment and the sound source number estimation technique described in the above-mentioned document 4. The third embodiment assumes that the true number of sound sources N ₀ is unknown but knows the upper limit thereof, and sets the assumed number of sound sources N to be larger than the upper limit of the true number of sound sources N ₀ .

(Configuration of Model Estimation Device According to Embodiment 3)
FIG. 4 is a diagram illustrating an example of the configuration of the model estimation apparatus according to the third embodiment. The model estimation device 10C according to the third embodiment further includes a sound source number estimation unit 13 as compared with the model estimation device 10B according to the second embodiment.

The number of sound sources estimation unit 13 uses the posterior probability γ ⁽ⁿ⁾ _tf calculated by the clustering unit 12B as the true sound source among the numbers n = 1 _,. Corresponding numbers n (1),..., N (N ₀ ) are determined, and only the parameters of the numbers corresponding to the true sound source are output. Specifically, the sound source number estimation unit 13 uses the posterior probability γ ⁽ⁿ⁾ _tf that the nth sound source is active to calculate the total posterior probability that the nth sound source is active, for example, (76) Calculated by the formula.

The sound source number estimation unit 13 clusters the total ρ ⁽ⁿ⁾ of posterior probabilities that each nth sound source is active into two, and the number n = n of ρ ⁽ⁿ⁾ belonging to the cluster with the larger sum. (1),..., N (^ N ₀ ) is obtained and regarded as the number corresponding to the true sound source. For example, the sound source number estimation unit 13 performs clustering by applying k-means clustering with two clusters to ρ ⁽ⁿ⁾ .

Finally, the sound source number estimation unit 13 outputs only the parameters shown in the following equation (77) corresponding to n = n (1),..., N (^ N ₀ ) corresponding to the true sound source. In the following (77) equation, l = 1, ···, is ^ N _0.

(Processing of model estimation apparatus according to Embodiment 3)
FIG. 5 is a flowchart illustrating an example of a processing procedure of the model estimation apparatus according to the third embodiment. The process of the model estimation device 10C described below is repeated until a predetermined convergence determination condition similar to that in the first or second embodiment is satisfied.

First, in step S21, the initialization unit 11B-1 calculates the initial value of the parameter set Θ based on the equations (47) to (49) and (70), and the model estimation device 10C Save to main storage. Next, in step S22, the dereverberation unit 11B-4, based on the regression matrix G _kf currently stored in the main storage device of the model estimation device 10C, _uses the above equation (56) to _{obtain a} mixed signal vector that does not include reverberation. Update the estimated value ^ x _tf of “(reverberation removal)”.

Next, in step S23, the posterior probability update unit 12B-1 has a posterior probability γ ^{(n (n} ) (n = 1,..., N) -th sound source signal is active at the time frequency point (t, f). ⁾ _tf is calculated by the above equation (71) and stored in the main memory of the model estimation device 10C. In step S23, the mixture weight updating unit 12B-2 calculates the mixture weight α ⁽ⁿ⁾ _t by the above equation (72) and stores it in the main storage device of the model estimation device 10C (hereinafter, “clustering”). processing). In step S23, the permutation resolution unit 12B-3 uses 共_f as a permutation on {1,..., N}, and the covariance matrix φ ^{( n)} _tf B ⁽ⁿ⁾ Replace _f .

  Next, the model estimation device 10C determines whether or not the convergence determination condition is satisfied (step S24). If the convergence determination condition is satisfied (Yes at Step S24), the model estimating apparatus 10C moves the process to Step S26. When the convergence determination condition is not satisfied (No at Step S24), the model estimating apparatus 10C moves the process to Step S25.

The process of step S25 is the same as the process of step S15 of the first embodiment shown in FIG. In step S26, the sound source number estimation unit 13 estimates the true number of sound sources using the posterior probability γ ⁽ⁿ⁾ _tf that the ^nth sound source is active, and outputs the estimation result.

[Embodiment 4]
The target sound enhancement apparatus according to the fourth embodiment is the target sound enhancement apparatus 100 including any of the model estimation apparatuses 10A to 10C according to the first to third embodiments.

(Configuration of target sound enhancement apparatus according to Embodiment 4)
FIG. 6 is a diagram illustrating an example of the configuration of the target sound enhancement device according to the fourth embodiment. The target sound enhancement apparatus 100 according to the fourth embodiment includes a frequency domain conversion unit 20, a model estimation apparatus 10A (or 10B or 10C), an enhancement sound calculation unit 30, and a time domain conversion unit 40.

The frequency domain transform unit 20 transforms the mixed signal vector to y _τ including reverberation in the time domain into a mixed signal vector y _tf including reverberation in the time frequency domain by time frequency conversion such as short-time Fourier transform. Here, the mixed signal vector ~y _tau, is defined by the following (78) below.

In the above equation (78), ˜y ^(m) _τ is a mixed signal including reverberation observed by the m (m = 1,..., M) th microphone in the time domain, and τ is Represents the sample number. The model estimation apparatus 10A (or 10B or 10C) calculates a set of parameters Θ and a posteriori probability γ ⁽ⁿ⁾ _tf that each sound source n is active.

The enhancement sound calculation unit 30 includes a mixed signal vector y _tf including reverberation in the time-frequency domain output from the frequency domain conversion unit 20, and a set of parameters Θ output from the model estimation device 10A (or 10B or 10C). Using the posterior probability γ ⁽ⁿ⁾ _tf that each sound source n is active, the estimated value ^ s ⁽ⁿ⁾ _tf of the microphone image not including the reverberation of each sound source in the time-frequency domain is expressed by the following equation (79) and (80) Calculate and output.

When the target sound enhancement apparatus 100 uses the model estimation apparatus 10A of Embodiment 1, the number n of γ ⁽ⁿ⁾ _tf is independent of the frequency prior to the processing of the above expressions (79) and (80). It is necessary to solve permutation so that it corresponds to the same sound source. This permutation solution is described in, for example, Reference 5 “H. Sawada, S. Araki, and S. Makino,“ Underdetermined convolutive blind source separation via frequency bin-wise clustering and permutation alignment. ”IEEE Trans. ASLP, vol. 19, no. 3, pp. 516.527, Mar. 2011. "

The time domain transforming unit 40 converts an estimated value of the microphone image that does not include the reverberation of each sound source in the time frequency domain output from the emphasized sound calculating unit 30 to the vector ^ s ⁽ⁿ⁾ _tf , such as inverse short-time Fourier transform. Applying the inverse of the time-frequency transform, a vector of estimated values of microphone images not including the reverberation of each sound source in the time domain ˜ ^ ⁽ⁿ⁾ _τ is calculated. Here, the vector ~ ^ s ⁽ⁿ⁾ _τ is defined by the following equation (81). Here, ~ ^ s ^{(m, n)} _τ is an inverse short-time Fourier transform of the m-th element ^ s ^{(m, n)} _tf of the vector ^ s ⁽ⁿ⁾ _τ .

Note that in the emphasized sound calculating section 30, an example of realizing dereverberation and source separation simultaneously, in order to remove the reverberation only, the estimated value ^ x _tf mixed signals without the reverberation in the time frequency domain Alternatively, an inverse transform of time-frequency transform such as inverse short-time Fourier transform may be applied to obtain a mixed signal estimated value vector ~ ^ _τ that does not include reverberation in the time domain. Here, the vector ~ ^ _xτ is defined by the following equation (82). Here, ~ ^ s ^{(m, n)} _τ is an inverse short-time Fourier transform of the m-th element ^ s ^{(m, n)} _tf of the vector ^ s ⁽ⁿ⁾ _τ .

(Processing of the target sound enhancement device according to the fourth embodiment)
FIG. 7 is a flowchart illustrating an example of a processing procedure of the target sound enhancement device according to the fourth embodiment. In the target sound enhancement apparatus 100 according to the fourth embodiment, first, in step S31, the frequency domain conversion unit 20 converts the signals observed by the respective microphones into signals in the time frequency domain. Next, in step S32, the model estimation apparatus 10A (or 10B or 10C) performs model estimation. Next, in step S33, the emphasized sound calculation unit 30 estimates the emphasized sound by calculation. Next, in step S34, the time domain conversion unit 40 converts the enhancement sound estimated by the enhancement sound calculation unit 30 from the frequency domain to the time domain.

  Hereinafter, examples of the disclosed embodiment and effects thereof will be described using the fourth embodiment as an example. 8 and 9 are diagrams for explaining an example of the effect of the fourth embodiment. Performance of the target sound enhancement apparatus 100 (hereinafter “proposed method”) according to the fourth embodiment and a conventional clustering-based sound source separation method that does not include a reverberation model (for example, the method described in Document 4, hereinafter “conventional method”) The experiment which compares was conducted. However, the model estimation device 10B according to the second embodiment is used as the model estimation device of the target sound enhancement device 100 according to the fourth embodiment.

A mixed signal including reverberation observed by a microphone was generated by convolving an impulse response (for example, see Reference 5 described above) measured in a laboratory with a speech waveform not including reverberation. FIG. 8 shows the positions of the microphone and the sound source when the impulse response is measured. In both the proposed method and the conventional method, prior to the estimation of the parameter Θ, the mixed signal vector y _tf including reverberation was whitened as shown in the above equations (59) to (61). The number N of sound sources is assumed to be known. Other experimental conditions were as shown in the following (Table 1). In addition, the laboratory shown in FIG. 8 was a space of 4.45 m × 3.55 m × (height) 2.50 m. Further, the height of Source 1 and 2 and Microphone 1 and 2 shown in FIG. 8 with respect to the floor of the laboratory was 1.2 m.

  The performance of the proposed method and the conventional method was evaluated by SIR (Signal-to-Interference Ratio) defined by the following equation (83).

Here, ˜ ^ s ^{(1, n, ν)} _τ represents the νth sound source component included in ˜ ^ s ^{(1, n)} . Τ: = 8 kHz × 8 s = 64000 represents the total number of sampling points, and _{Σν ≠ n} represents the sum for values of ν other than n.

Here, how to obtain ~ ^ s ^{(1, n, ν)} _τ will be described. The mixed signal vector y _tf including the observed reverberation can be decomposed as the following equation (84) using the microphone image vector x ^(ν) _tf including the reverberation of the νth sound source.

Therefore, the estimated vector ^ s ⁽ⁿ⁾ _tf of the microphone image that does not include the reverberation of the nth sound source can be decomposed as shown in the following equations (85) and (86).

Here, in the above equation (86) ^{, ｓs (n, ν)} _tf represents the _νth sound source component included in _ｓs ⁽ⁿ⁾ _tf . Therefore, by the following (87) equation, ^ s ^{(n, [nu)} seeking _tf, seek ^ s ^{(n, [nu)} ~ and inverse short time Fourier transform ^{_{tf ^ s (n, ν)}} τ, ~ ^ s ^{(n, ν)} ~ as the first element of ^{_{τ ^ s (1, n,}} ν) τ is obtained.

  FIG. 9 shows a graph in which the average value of SIR is plotted for each reverberation time when eight trials are performed with different combinations of speech waveforms. Under the conditions with the shortest reverberation time (reverberation time of about 130 ms), the proposed method and the conventional method showed equivalent performance. However, as shown in FIG. 9, as the reverberation time increases, the performance improvement amount of the proposed method with respect to the conventional method tends to increase. In particular, when the reverberation time was about 370 ms, the performance improvement amount was about 4 dB at the maximum during the trial.

  As described above, the reverberation based on linear prediction is used in the first to fourth embodiments in the sound source separation technology based on clustering that has an advantage that it can be applied even when the number of sound sources is unknown, compared to the sound source separation technology based on independent component analysis. The sound source separation based on removal and clustering is repeated alternately. Embodiments 1 to 4 can improve sound source separation performance by applying sound source separation based on clustering to a mixed signal that does not include reverberation estimated by dereverberation based on linear prediction. Furthermore, Embodiments 1 to 4 can improve the performance of dereverberation by using the improved sound source separation result. Thus, Embodiments 1 to 4 can realize more accurate sound source separation even when the reverberation time is longer than the frame length by repetitive reverberation removal and sound source separation.

(About the device configuration of the model estimation device and the target sound enhancement device)
The components of the model estimation devices 10A to 10C shown in FIGS. 1, 3, and 4 and the target sound enhancement device 100 shown in FIG. 6 are functionally conceptual and are not necessarily physically configured as shown. I don't need to be. That is, the specific forms of distribution and integration of the functions of the model estimation apparatuses 10A to 10C and the target sound enhancement apparatus 100 are not limited to those shown in the drawings, and all or a part of them can be arbitrarily selected according to various loads and usage conditions. It can be configured to be functionally or physically distributed or integrated in units.

  In addition, each of the processes performed in the model estimation apparatuses 10A to 10C and the target sound emphasizing apparatus 100 is entirely or arbitrarily partly performed by a processing apparatus such as a CPU (Central Processing Unit) and a program executed by the processing apparatus. It may be realized. Moreover, each process performed in model estimation apparatus 10A-10C and the target sound emphasis apparatus 100 may be implement | achieved as hardware by a wired logic.

  In addition, among the processes described in the embodiment, all or a part of the processes described as being automatically performed can be manually performed. Alternatively, all or some of the processes described as being manually performed among the processes described in the embodiments can be automatically performed by a known method. In addition, the above-described and illustrated processing procedures, control procedures, specific names, and information including various data and parameters can be changed as appropriate unless otherwise specified.

(About the program)
FIG. 10 is a diagram illustrating an example of a computer in which a model estimation device and a target sound enhancement device are realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. In the computer 1000, these units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to the display 1061, for example.

  The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the model estimation devices 10A to 10C and the target sound enhancement device 100 is stored in, for example, the hard disk drive 1031 as a program module 1093 in which a command executed by the computer 1000 is described. For example, a program module 1093 for executing information processing similar to the functional configuration in the model estimation devices 10A to 10C and the target sound enhancement device 100 is stored in the hard disk drive 1031.

  The setting data used in the processing of the embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1031 to the RAM 1012 as necessary, and executes them.

  Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1031, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 via the network interface 1070.

  The above-described embodiments and other embodiments are included in the invention disclosed in the claims and equivalents thereof as well as included in the technology disclosed in the present application.

10A, 10B, 10C Model estimation apparatus 11A, 11B Reverberation removal processing unit 11A-1, 11B-1 Initialization unit 11A-2, 11B-2 Covariance matrix update unit 11A-3, 11B-3 Regression matrix update unit 11A- 4, 11B-4 Reverberation removal unit 12A, 12B Clustering unit 12A-1, 12B-1 A posteriori probability update unit 12A-2, 12B-2 Mixed weight update unit 12B-3 Permutation resolution unit 13 Sound source number estimation unit 20 Frequency Region conversion unit 30 Emphasized sound calculation unit 40 Time region conversion unit 100 Target sound enhancement device 1000 Computer 1010 Memory 1020 CPU

Claims (8)

A storage unit for storing parameters of a model of a mixed signal including reverberation including a regression matrix indicating characteristics of reverberation due to sounds output from a plurality of sound sources;
A signal estimation unit for estimating a mixed signal not including the reverberation by linear prediction using an observation signal obtained by observing the sound with a plurality of microphones and a regression matrix stored in the storage unit;
A posterior probability calculation unit that calculates a posterior probability for each cluster corresponding to the sound source to which each time frequency point belongs, from the mixed signal estimated by the signal estimation unit;
The parameter is estimated and estimated from the observed signal, the mixed signal estimated by the signal estimation unit, the posterior probability calculated by the posterior probability calculation unit, and the parameter stored in the storage unit. A parameter estimation unit that updates a parameter stored in the storage unit with a parameter, and
The signal estimation unit, the posterior probability calculation unit, and the parameter estimation unit repeat each process until a predetermined condition is satisfied. The mixed signal model including reverberation is a probabilistic model representing a distribution of the mixed signal including reverberation;
The probability model is a mixture model represented by a weighted sum of probability models representing a distribution of a mixture signal including the reverberation for each of the clusters;
The model estimation apparatus according to claim 1, wherein the parameter estimation unit estimates the parameter using a predetermined evaluation function for evaluating the probability model. The predetermined evaluation function is a likelihood of a mixed signal including the reverberation with respect to a parameter estimated by the parameter estimation unit, or a posterior probability of a parameter estimated by the parameter estimation unit. 2. The model estimation apparatus according to 2. The parameter estimated by the parameter estimation unit includes a mixing weight value indicating a distribution of the plurality of sound sources included in the mixed signal including the reverberation at each time frequency point,
The model estimation apparatus according to claim 3, wherein the mixing weight value is a mixing weight value for each frequency of the mixed signal including the reverberation or a mixing weight value for each time of the mixed signal including the reverberation. The parameter estimation unit, based on the posterior probability corresponding to each of the plurality of sound sources included in the mixed signal including the reverberation at each time frequency point, the sound source included in the mixed signal including the reverberation among the plurality of sound sources. The model estimation apparatus according to claim 4, wherein a parameter corresponding to the estimated sound source is used as the estimated parameter. From the parameter and the posterior probability estimated by the model estimation device according to any one of claims 1 to 5 and a mixed signal including reverberation of each sound source in the time frequency domain, in the time frequency domain. An objective sound emphasizing apparatus comprising: an output unit that estimates and outputs an estimated value of an acoustic signal that does not include reverberation of each of the sound sources. A model estimation method executed by a model estimation device,
The model estimation apparatus includes a storage unit that stores a parameter of a model of a mixed signal including reverberation including a regression matrix indicating characteristics of reverberation due to sound output from a plurality of sound sources,
A signal estimation step of estimating a mixed signal not including the reverberation by linear prediction using an observation signal obtained by observing the sound with a plurality of microphones and a regression matrix stored in the storage unit;
A posterior probability calculation step of calculating a posterior probability for each cluster corresponding to the sound source to which each time frequency point belongs, from the mixed signal estimated by the signal estimation step;
The parameter is estimated and estimated from the observed signal, the mixed signal estimated by the signal estimation step, the posterior probability calculated by the posterior probability calculation step, and the parameter stored in the storage unit. A parameter estimation step of updating a parameter stored in the storage unit with a parameter, and
The signal estimation step, the posterior probability calculation step, and the parameter estimation step are repeated until a predetermined condition is satisfied.   A model estimation program for causing a computer to function as the model estimation apparatus according to claim 1.

Images