JP2012173592A - Sound source parameter estimation device and sound source separation device and method thereof and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound source parameter estimation device capable of estimating a sound source model parameter in conjunction with a sound source parameter even when no sound source model parameter is given in advance.SOLUTION: A sound source model parameter update section updates a sound source model parameter using a sound source power feature amount, a sound source power parameter, a sound source occupancy, a preliminary probability density function of the sound source power parameter stored in a sound source model storage section, and a model of the sound source power feature amount as inputs. A sound source occupancy update section updates a sound source occupancy for each sound source using a sound source position feature amount, the sound source power feature amount, the updated sound source power parameter for each sound source, a sound source position parameter, the sound source model parameter, the preliminary probability density function of the sound source power parameter stored in the sound source model storage section, and a model of the sound source power feature amount as inputs.

Description

  The present invention relates to a sound source parameter estimation device that estimates sound source parameters of each sound source from observation signals collected by a plurality of microphones mixed with acoustic signals generated simultaneously by a plurality of sound sources, and each sound source based on the sound source parameters The present invention relates to a sound source separation device that separates sound sources and methods and programs thereof.

  FIG. 7 shows a functional configuration of a conventional sound source parameter estimation apparatus 900 disclosed in Non-Patent Document 1. The sound source parameter estimation apparatus 900 includes a sound source model storage unit 920, a feature extraction unit 910, a sound source power parameter update unit 930, a sound source position parameter update unit 940, and a sound source occupancy update unit 950.

The sound source model storage unit 920 includes a prior probability density function p (q ^(l) ) ((l) is the lth sound source) of sound source power parameters representing the state of the entire sound source power time series of each of a plurality of sound source signals, Sound source power feature model f = β _{q (l), n, k,} which is a posterior probability density function at each time frequency point (n, k) of each sound source signal when the sound source power parameter q ^(l) is given (S) is memorized. The feature extraction unit 910 receives an observation signal x ^(m) _{n, k obtained} by converting a time domain signal obtained by collecting a plurality of sound source signals with a plurality (m) of microphones into a time frequency domain signal, and inputs each time frequency. A sound source position feature amount _{An, k} and a sound source power feature amount X _{n, k at a point} are extracted.

The sound source power parameter update unit 930 is a sound source occupancy ^ M ⁽ which is an a posteriori probability density function of an exclusive sound source when the sound source power feature amount X _{n, k} and the observation signal are obtained for each of N _s sound sources. ^{l) Using} _{n, k} , sound source power feature model f = β _{q (l), n, k} (S) and prior probability density function p (q ^(l) ) of sound source power parameters as input, Update parameter ^ q ^(l) . The sound source position parameter update unit 940 receives the sound source position feature quantity _{An, k} and the sound source occupancy ^ M ^(l) _{n, k} and updates the sound source position parameter ^ φ ^(l) of each sound source.

The sound source occupancy update unit 950 includes a sound source position feature amount _{An, k} , a sound source power feature amount X _{n, k} , an updated sound source power parameter ^ q ^(l) and a sound source position parameter ^ φ ^{(l) of} each sound source. , The sound source occupancy ^ M ⁽ of each sound source based on the model f = β _{q (l), n, k} (S) and the prior probability density function p (q ^(l) ) of the sound source power parameter ^l) Update _{n, k} .

  The sound source separation apparatus using the sound source parameter estimation technique of the sound source parameter estimation apparatus 900 includes a sound source power feature amount output from the sound source parameter estimation apparatus 900, an updated sound source occupancy and a sound source power parameter, and each time frequency of each sound source signal. A sound source separation unit that obtains a sound source separation signal of each of a plurality of sound sources by least square error estimation using a model of a sound source power feature amount at the point as an input is further provided.

Tomohiro Nakatani, Shoko Araki, Takuya Yoshioka, Masakiyo Fujimoto, “Multichannel source separation based on source location cue with log-spectral shaping by hidden Markov source model,” Proc. Of Interspeech-2010, pp. 2766-2769, Sep., 2010 . Tomohiro Nakatani, Akiko Araki, Takuya Yoshioka, Masaki Fujimoto, “Sound source separation based on DOA clustering and logarithmic spectrum HMM of speech” Proc. .

  In a conventional sound source parameter estimation device, a sound source model parameter that is a parameter for controlling the respective behaviors of the sound source power parameter a priori probability density function stored in the sound source model storage unit and the sound source power feature amount model is not given in advance. Sound source parameters could not be estimated.

  The present invention has been made in view of this problem, and even when no sound source model parameter is given in advance, it is possible to estimate a sound source parameter as well as other sound source parameters as a part of the sound source parameter. It is an object to provide a device, a sound source separation device, and a method and program thereof.

  A sound source parameter estimation apparatus according to the present invention includes a sound source model storage unit, a feature extraction unit, a sound source power parameter update unit, a sound source position parameter update unit, a sound source model parameter update unit, and a sound source occupancy degree update unit. To do. The sound source model storage unit is a prior probability density function of a sound source power parameter representing the state of the entire sound source power time series of each of the plurality of sound source signals, and each time frequency point of each sound source signal when the sound source power parameter is given. The sound source power feature amount model is stored. The feature extraction unit extracts the sound source position feature value and the sound source power feature value at each time frequency point by using the observation signal obtained by converting the time domain signal obtained by collecting multiple sound source signals with multiple microphones into the time frequency domain signal. To do. The sound source power parameter update unit includes a sound source power feature amount, a sound source occupancy that is an a posteriori probability density function of the exclusive sound source obtained from the observed signal, a prior probability density function of the sound source power parameter, and each sound source signal. The sound source power parameters of each sound source are updated with the sound source power feature model and the sound source model parameters that control the behavior of the sound source power feature model and the prior probability density function of the sound source power parameters as inputs. To do. The sound source position parameter update unit receives the sound source position feature amount and the sound source occupancy as input, and updates the sound source position parameter of each sound source. The sound source model parameter update unit receives the sound source power feature amount, the sound source power parameter, the sound source occupancy, the prior probability density function of the sound source power parameter stored in the sound source model storage unit, and the model of the sound source power feature amount as inputs. Update parameters. The sound source occupancy degree update unit pre-determines the sound source position feature amount, the sound source power feature amount, the updated sound source power parameter, the sound source position parameter, the sound source model parameter, and the sound source power parameter stored in the sound source model storage unit in advance. The sound source occupancy of each sound source is updated with the probability density function and the model of the sound source power feature quantity as inputs.

  According to the sound source parameter estimation device of the present invention, the sound source model parameters can be estimated together with other sound source parameters as part of the sound source parameters even when the sound source model parameters are not given in advance. It is possible to give optimal sound source parameters to various sound sources that cannot be given statistical properties.

  The sound source separation device of the present invention using the sound source parameter estimation device can output a sound source separation signal with less error.

The figure which shows the function structural example of the sound source parameter estimation apparatus 100 of this invention. The figure which shows the operation | movement flow of the sound source parameter estimation apparatus 100. The figure which shows the function structural example of the sound source separation apparatus 200 of this invention. The figure which shows the operation | movement flow of the sound source separation apparatus. It is a figure which shows an experimental result, (a) is a figure which shows the result estimated from the audio | voice before mixing, (b) is a figure which shows the result estimated from the mixed sound. It is a figure which shows an experimental result, and shows the relationship between the cepstrum distortion of the signal after isolation | separation, and the number of mixed sounds. These are figures which show the result of having processed with respect to the mixed sound (mixing conditions 2 and 3) recorded in another environment with reverberation. The figure which shows the function structure of the conventional sound source parameter estimation apparatus 900.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. Prior to the description of the embodiments, the basic idea of the present invention will be described.

[Basic idea of the present invention]
In the present invention, in order to enable the estimation of the sound source parameter even if the sound source model parameter is not given in advance, the entire time series of the sound source power feature amount of the l-th sound source signal is represented by {S ^(l) _{n, k} }. The simultaneous probability density function is modeled including the sound source model parameter ψ ^(l) , and each sound source power feature amount X _{n, k of} a signal in which multiple sounds are mixed is given. It is new in that it has a sound source occupancy degree, a model of sound source power features, a prior probability density function of sound source power parameters, and a sound source model parameter update unit that updates the sound source model parameters of each sound source based on the sound source power parameters.

First, symbols used for description will be described. The observed signal is superimposed N _s number of sound source signals, picks up the sound signal in N _m the microphones. An observation signal obtained by converting a collected signal collected from the m-th microphone into a frequency domain signal using a short-time Fourier transform or the like is denoted as X ^(m) _{n, k} . n is the nth time or frame number, k is the kth frequency or bin number, and when referring to the time frequency point corresponding to the nth time and the kth frequency, the time frequency point (n, k ). Note that the position of the symbol ^ and the notation of the subscript and its position are correct in the expression.

In the present invention, the simultaneous probability density function of the entire sound source power time series {S ^(l) _{n, k} } of the l-th sound source signal is modeled as shown in the following equation.

Here, q ^(l) represents a sound source power parameter representing the state of the entire sound source power time series of the l-th sound source. In the following, q ^(l) of all sound sources is collectively expressed as q = [q ⁽¹⁾ , ..., q ^(Ns) ]. ψ ^(l) represents the entire sound source model parameter of the l-th sound source. Ψ ^(l) of all sound sources is collectively expressed as ψ = [ψ ⁽¹⁾ ,..., Ψ ^(Ns) ].

Β _{q (l), n, k, ψ (l)} (S) is a model of the sound source power feature, and given the sound source power parameter q ^(l) and the sound source model parameter ψ ^(l) , This is a probability density function in which the sound source power of the sound source signal at each time frequency point (n, k) is S ^(l) _{n, k} (formula (3)). Summation of equation (1), if q ^(l) takes continuous values rather than discrete values shall be expressed by replacing the integral operation on q ^(l). In the equation (2), the assumption is made that the sound source powers S ^(l) _{n, k at} different time frequency points are independent from each other when the state of the sound source is known.

In the present invention, as shown in the equation (4), the sound source power S ^(l) _n of the sound source signal having the largest energy at each time frequency point (n, k) (hereinafter referred to as an exclusive sound source signal ^). _{, k} is assumed to match the sound source power of the observed signal.

Further, it is assumed that the non-occupying sound source 1 has a relationship of S ^(l) _{n, k} ≦ X _{n, k} . Then, it is known that the posterior probability density function of the sound source power X _{n, k} of the observed signal can be expressed as follows under the condition that the state of each sound source signal is known (reference: SJ Rennie, JR Hershey , and PA01sen, “Hierarchical variational loopy belief propagation for multi-talker speech recognition,” Proc. ASRU-2009, pp. 176-181 2009.).

  In the present invention, it is further assumed that the above equation can be decomposed as follows.

In the present invention, in order to estimate the sound source position parameter φ ^ ^(l) from the sound source position feature quantity, a sound source position feature quantity model p (A _{n, k} ; φ) is introduced. The sound source location feature model p (A _{n, k} ; φ) assumes that the energy of each sound source signal is sparsely distributed over different time frequency points, and the sound source location of the sound source that is occupied at that time frequency point Suppose that it depends only on.

Generally, when the sound source position parameters φ ^(l) of all sound sources are collectively expressed as φ = [ψ ⁽¹⁾ ,..., Ψ ^(Ns) ], the sound source position feature model p (A _{n, k} ; φ), that is, the probability density function of the sound source position feature quantity of the observation signal can be developed as a mixture distribution as shown in Expression (8).

In Equation (8), Z _{n, k} is a random variable that represents the number of the sound source that is occupied at the time frequency point (n, k), and Z _{n, k} = l is a sound source that is occupied by the l-th sound source. The case is shown. P (Z _{n, k} = l) represents a prior probability density function in which the l-th sound source becomes an exclusive sound source at the time frequency point (n, k). Further, the following notation is used in the following description.

γ _{φ (l), n, k} (A) is the probability density function that gives the sound source position feature quantity An _{n, k} when the number of the sound source occupied at the time frequency point (n, k) is l. To express. This depends only on the sound source position parameter φ ^(l) of the l-th sound source. Specific definitions of γ _{φ (l), n, k} (A) and φ ^(l) will be described later.

If γ _{φ (l), n, k} (A) is defined under equation (8), the prior probability density function p (Z _n regarding the sound source position parameter φ ^(l) and the number of the occupied sound source _{, k} = l), the sound source position feature quantity model p (A _{n, k} ; φ) can be uniquely determined. Conversely, when the sound source position feature quantity An _{n, k} is observed, the prior probability density function p (Z _{n, k} = l) for the sound source position parameter and the number of the occupied sound source according to a method such as maximum likelihood estimation And its posterior probability density function.

  According to the above definition, the probability density function of complete data is derived as shown in equation (10).

  In equation (10), q is a sound source power parameter, ψ is a sound source model parameter, and φ is a sound source position parameter, and these parameters are parameters to be estimated. In the present invention, the sound source power parameter q, the sound source model parameter ψ, and the sound source position parameter φ are estimated as values that maximize the next log likelihood function.

In equation (12), the random variable Z _{n, k} is treated as a hidden variable. For example, an expectation maximization algorithm can be used to maximize the log likelihood function including hidden variables. In the expected value maximization algorithm, the sound source power parameter estimate ^ q, the sound source position parameter estimate ^ φ and the sound source model parameter estimate ^ ψ The a posteriori probability density function ^ M ^(l) _{n, k} = p (Z _{n, k} │A _{n, k} , X _{n, k} , ^ q; ^ φ, ^ ψ) must be estimated at the same time. In the present invention, the value of this function is referred to as the sound source occupancy, and this value is also included in the sound source parameters.

In the above-described concept, both the sound source power feature model β _{q (l), n, k, ψ (l)} (S) and the sound source position feature model p (A _{n, k} ; φ) are considered. However, the estimation error of the sound source parameter can be reduced by estimating the optimum sound source parameter. In addition, the variable Z _n representing the number of the exclusive sound source is added to the sound source position feature quantity model p (A _{n, k} ; φ) (formula (8)) and the sound source power feature quantity model (formula (7)). _{, k} can be used to simplify calculation of sound source parameter estimation while considering two feature quantities.

As described above, according to the sound source parameter estimation method of the present invention, the simultaneous probability density function of the entire sound source power time series {S ^(l) _{n, k} } of the l-th sound source signal is expressed as the sound source model parameter ψ ^{(l )} with model including, excitation power characteristic quantity X _n of a signal in which a plurality of sound are _mixed, when _{the k} is given, the sound source occupancy of each sound source, the sound source power feature quantity model and the excitation power parameter By including a sound source model parameter updating unit that updates the sound source model parameters of each sound source based on the prior probability density function and the sound source power parameters, the sound source parameters can be estimated even if the sound source model parameters are not given in advance.

FIG. 1 shows a functional configuration example of a sound source parameter estimation apparatus 100 of the present invention. The operation flow is shown in FIG. The sound source parameter estimation apparatus 100 includes a feature extraction unit 910, a sound source model storage unit 90, sound source power parameter update units 60 _{1 to} 60 _Ns corresponding to the number of sound sources, and sound source power parameter update units 60 _{1 to} 60 _Ns. Includes the same number of sound source position parameter update units 940 _{1 to} 940 _Ns , a sound source occupancy update unit 70, and a sound source model parameter update unit 80. The functions of the respective units are realized by a predetermined program being read into a computer constituted by, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

The sound source parameter estimation device 100 is different from the sound source power parameter estimation device 900 described in the related art in that a sound source model parameter update unit 80 is provided. The feature extraction unit 910 and the sound source position parameter update units 940 _{1 to} 940 _Ns are the same as those of the sound source power parameter estimation apparatus 900 as is apparent from the reference symbols.

  The operation of each functional unit will be described.

(Feature extraction unit)
The feature extraction unit 910 receives an observation signal x ^(m) _{n, k obtained} by converting a time domain signal obtained by collecting a plurality of sound source signals with a plurality of (m) microphones into a time frequency domain signal, and inputs each time frequency point. A sound source position feature value An _{n, k} and a sound source power feature value X _{n, k} at (n, k) are extracted.

For example, when the logarithmic power spectrum of the signal collected by the first microphone is extracted as the sound source power feature amount, the sound source power feature amount X _{n, k} is calculated as shown in Expression (13). The sound source power feature amount X _{n, k} is input to the sound source occupancy update unit 70 and the sound source power parameter update units 60 _{1 to} 60 _Ns .

The sound source position feature amount _{An, k} generally appears in the phase difference or intensity ratio of signals between different microphones at each time frequency point. Therefore, the sound source position feature amount _{An, k} is a vector that can be obtained by collecting signal phase differences and intensity ratios for different microphone pairs, or is extracted as a value obtained as a result of some feature extraction. For example, when a phase difference between signals collected by two microphones is extracted as the sound source position feature amount _{An, k} , the calculation is performed as shown in Expression (14).

[Sound source model storage unit]
The sound source model storage unit 90 includes a prior probability density function p (q ^(l) ; ψ ^(l) ) of a sound source power parameter q ^(l) representing the state of the entire sound source power time series of each of a plurality of sound source signals, and the sound source A model β _{q (l), n, k, ψ (l)} (S) of the sound source power feature quantity at each time frequency point of each sound source signal when the power parameter q ^(l) is given is stored. (S) is a variable representing the sound source power feature amount X _{n, k} .

[Sound source occupancy update section]
The sound source occupancy update unit 70 includes the sound source position feature quantity An _{n, k} , the sound source power feature quantity X _{n, k} , the updated sound source power parameter ^ q ^(l) and the sound source position parameter ^ φ ^{(l) of} each sound source. , The sound source model parameter ^ ψ ^(l) , the prior probability density function p (q ^(l) ; ψ ^(l) ) of the sound source power parameter stored in the sound source model storage unit 90, and the model β _{q ( l), n, k, ψ (l)} (S) are input and the sound source occupancy of each sound source is updated.

The sound source occupancy update unit 70 initializes the sound source occupancy ^ M ^(l) _{n, k} with, for example, random numbers so that Σ _l ^ M ^(l) = 1 (step S70). Or you may use the same method as the sound source parameter estimation apparatus 900 demonstrated by the prior art. Thereafter, the processes of the sound source power parameter update units 60 _{1 to} 60 _Ns , the sound source occupation rate update unit 70, the sound source position parameter update units 940 _{1 to} 940 _Ns, and the sound source model parameter update unit 80 are repeated until they converge.

[Sound source power parameter update unit]
The sound source power parameter updating units 60 _{1 to} 60 _Ns store the sound source power feature quantity X _{n, k} , the sound source occupation degree ^ M ^(l) _{n, k} initialized for each sound source l _, and the sound source model storage unit 90. Prior probability density function p (q ^(l) ; ψ ^(l) ) of stored sound source power parameter ^ q ^(l) and model β _{q (l), n, k, ψ (l)} ( Using S) and the sound source model parameter ^ ψ ^(l) as inputs, the sound source power parameter ^ q ^(l) is updated (M-step) as shown in equation (15) (step S60).

[Sound source position parameter update unit]
The sound source position parameter updating units 940 _{1 to} 940 _Ns receive the sound source position parameter ^ φ by inputting the sound source occupancy ^ M ^(l) _{n, k} and the sound source position feature quantity An _{n, k} initialized for each sound source l. ^(l) is updated (M-step) as shown in equation (17) (step S940).

[Sound source model parameter update unit]
The sound source model parameter update unit 80 includes sound source power feature amount X _{n, k} , sound source power parameter ^ q ^(l) , sound source occupancy ^ M ^(l) _{n, k} , and sound source power stored in the sound source model storage unit 90. The sound source model parameter ^ using the parameter prior probability density function p (q ^(l) ; ψ ^(l) ) and the sound source power feature model β _{q (l), n, k, ψ (l)} (S) as input ψ ^(l) is updated as shown in equation (15 ′) (step S81).

The sound source occupancy update unit 70 then updates the sound source power parameter ^ q ^(l) , the sound source model parameter ^ ψ ^(l) , the sound source position feature quantity An _{n, k,} and the sound source power feature quantity X updated for each sound source l. _{n, k} , the prior probability density function p (q ^(l) ; ψ ^(l) ) of the sound source power parameter stored in the sound source model storage unit 90, and the model β _{q (l), n, k of the} sound source power feature quantity _{, ψ (l)} (S) as inputs, and the sound source occupancy ^ M ^(l) _{n, k} is updated (E-step) as shown in equation (18) (step S71).

  Steps S60 to S71 are repeated until convergence is obtained (no in step S72). A second embodiment using a more specific model of the sound source position feature amount and the model of the sound source power feature amount will be described next.

First, the feature extraction unit 910 extracts the inter-microphone phase difference as the sound source position feature quantity _{An, k} based on the equation (14). Further, it is assumed that the phase difference between the microphones of the observation signal derived from each sound source l follows a Gaussian distribution having different mean values μ ^(l) _k and variances σ ^(l) _k for each frequency. Equation (9) can then be defined as follows:

Where φ ^(l) _k = [μ ^(l) _k , σ ^(l) _k ] is a part of the sound source position parameter φ ^(l) extracted only for frequency k, and φ ^(l) is all Φ ^(l) = [φ ^(l) ₁ ,..., Φ ^(l) _Nk ] in which φ ^(l) _k is collected with respect to frequency k. N (•) represents a probability density function of Gaussian distribution.

On the other hand, the feature extraction unit 910 extracts the logarithmic power spectrum of any one of the microphone signals as the sound source power feature amount X _{n, k} based on the equation (13). Furthermore, the sound source power parameter q ^(l) is decomposed into a state sequence representing the state at each time as q ^(l) = {q ^(l) ₀ , q ^(l) ₁ , ...}, and a first-order Markov process And state transitions occur at each time.

However, the sound source power parameter q ^(l) ₀ represents the initial state of the hidden Markov model. Further, the posterior probability density function of S ^(l) _{n, k} at each time frequency point (n, k) defined by Equation (3) follows a Gaussian distribution that depends only on the state q ^(l) _{n at} that time. Assume that This is expressed by the following formula.

Where π ^(l) _i = p (q ^(l) ₀ = i) is the prior probability that the initial state of the hidden Markov model is i, α ^(l) _{i, j} = p (q ^(l) _n = j | q ^(l) _n-1 = i) is the state transition probability that the hidden Markov model moves from state i to state j, β _{i, n, k, ψ (l)} (S) = p (S ^(l) _{n, k} = S | q ^(l) _n = i; ψ ^(l) ) = N (S ^(l) _{n, k} ; μ ^(l) _{i, k} , σ ^(l) _{i, k} ) is a hidden Markov The probability density function of the output in state i of the model, and μ ^(l) _{i, k} and σ ^(l) _{i, k} are the mean and variance.

Based on this definition, the sound source model parameter ^ ψ ^(l) is π ^(l) _i , α ^(l) _{i, j} , μ ^(l) _{i, k} , σ ^(for all i, j, k, l ^l) It consists of _{i and k} . In this embodiment, all or some of the sound source model parameters are estimated by the expected value maximization algorithm as part of the sound source parameters.

Under the above assumptions, M-step1 of the expected value maximization algorithm already explained in FIG. 2 is a state time series for which the sound source power parameter updating units 60 _{1 to} 60 _Ns satisfy Expression (22) for each sound source l Update q ^(l) = [^ q ^(l) ₀ , ..., ^ q ^(l) _Ns ] using the Viterbi algorithm.

In addition, for each sound source l, the sound source position parameter update units 940 _{1 to} 940 _Ns have φ ^(l) _k = [μ ^(l) _k , σ ^(l) _k ] at all frequencies _k . Is updated as follows.

In M-step 3, the sound source model parameter update unit 80 updates the sound source model parameter ^ ψ ^(l) for each sound source l. First, if π ^(l) , α ^(l) is a set of π ^(l) _i , α ^(l) _{i, j related to i, j} , then π ^(l) , α ^(l) is updated as follows: Is done.

The above update can be easily realized by a known method for learning a hidden Markov model. On the other hand, if μ ^(l) , σ ^(l) is a set of μ ^(l) _{i, k} , σ ^(l) _{i, k} for all _{i, k} , the update of μ ^(l) , σ ^(l) is This is realized as follows.

As one method for realizing the above update, a sequential maximization algorithm represented by a quasi-Newton method, a conjugate gradient method, or the like can be raised. Many generally known algorithms can be applied to this. On the other hand, log (β _{q (l) n, n, k, ψ (l)} (X _{n, k} )) included in the above function is easy to handle analytically, but log (ρ _{q (l ) n, n, k, ψ (l)} (X _{n, k} )) contains integral operations, making it more complicated to handle analytically and making the calculations required for sequential maximization algorithms relatively complex There is a problem that makes it. One way to avoid this is to approximate log (ρ _{q (l) n, n, k, ψ (l)} (X _{n, k} )) with a function that is relatively easy to handle analytically. . In the following, this will be described in some detail.

First, set x = (X _{n, k} -μ ^(l) _{i, k} ) (σ ^(l) _{i, k} ) ^-1/2 and f (x) = log (ρ _{q (l) n, n,} Rewrite the following regular expression equivalent to _{k, ψ (l)} (X _{n, k} )).

  Then, the first derivative df (x) / dx of the function f (x) can be approximated by one of the following functions.

Both of these functions are easy to handle analytically. In particular _, when used to update μ ^(l) _{i, k} , σ ^(l) _{i, k} , for example, the following two advantages can be obtained. it can.

The first is Newton's method, which is one of the iterative estimation methods that can obtain fast convergence by approximating the _first derivative of f (x) with g ₁ (x) for the update of μ ^(l) _{i, k} The global convergence is guaranteed. The second is that g ₂ (x) has the same mathematical form as the first derivative of log (ρ _{q (l) n, n, k, ψ (l)} (X _{n, k} )). , Μ ^(l) _{i, k} , σ ^(l) _{i, k} update by approximating the first derivative of f (x) with g ₂ (x) Algorithms can be applied.

Therefore, for example, when g ₁ (x) is used, one update of μ ^(l) _{i, k} by the Newton method can be realized as follows.

Here, ν ^(l) _i represents a set of time n satisfying q ^(l) _n = i. g ₁ ′ (x) represents the first derivative of g ₁ (x).

On the other hand, using g ₂ (x) _, the update of σ ^(l) _{i, k} can be realized as follows.

However, κ ^(l) _{i, k} assumes the following values.

  Further, the E-step performed by the sound source occupancy update unit 70 updates the sound source occupancy as shown in Expression (32).

Here, another method for initializing the sound source model parameter ^ ψ ^(l) when there is no previously learned hidden Markov model will be described. The time series of sound source power features is modeled with a mixed Gaussian model, and the resulting mixture parameters α ′ to π ^(l) and α ^(l) of the distribution parameters ψ ′ of the mixed Gaussian distribution are Μ ^(l) _{i, k} and σ ^(l) _{i, k} are determined from the average μ ′ _{i, k} and the variance σ ′ _{i, k} . At this time, the mixed Gaussian model takes the following form.

In order to determine the distribution parameters of the mixed Gaussian distribution under the condition where the sound source power feature amount X _{n, k} is given, for example, a generally known method such as an expected value maximization algorithm is applied. can do. Based on the distribution parameters obtained as a result, the sound source model parameter ^ ψ ^(l) can be determined as follows.

  As a result, it is possible to initialize the sound source model parameter that approximately represents the distribution of the observed sound source power feature quantity.

[Modification 1]
Next, a modified example of a method for updating μ ^(l) _{i, k} which is one of the sound source model parameters ψ ^(l) updated by the sound source model parameter updating unit 80 will be described. For _example, μ ^(l) _{i, k} of the a priori probability density function _{^{p (μ (l) i,}} k) = N (μ (l) i, k; ~ μ (l) i, k, ~ σ (l) i _{, k} ) is given. As the distribution parameter of this prior probability density function, the value of the sound source model parameter initialized based on the method described in the example described above is used, and ~ μ ^(l) _{i, k} = ^ μ ^{(l )} _{i, k} , ~ σ ^(l) _{i, k} = ^ σ ^(l) It has been confirmed by experiments that it is effective to define _{i, k} .

Using this prior probability density function _, the update equation (29) for ^ μ ^(l) _{i, k} is modified as follows based on the posterior probability maximization criterion.

  Here, ρ is a control parameter (> 0) for adjusting the weight of the prior probability density function, and can be freely determined based on the degree to which the prior probability density function is trusted.

[Sound source separation device]
FIG. 3 shows a functional configuration example of the sound source separation device 200 of the present invention. The operation flow is shown in FIG. The sound source separation device 200 includes the above-described sound source parameter estimation device 100 and a sound source separation unit 95.

The sound source separation unit 95 outputs the sound source power feature amount X _{n, k} output from the sound source parameter estimation device 100, the updated sound source occupancy ^ M ^(l) _{n, k} , the sound source power parameter ^ q ^(l) _n, and the sound source model. Each of a plurality of sound sources is input with the parameter ^ ψ ^(l) and the model β _{q (l), n, k, ψ (l)} (S) of the sound source power feature quantity at each time frequency point of each sound source signal. The sound source separation signal ^ S ^(l) _{n, k} is _obtained by least square error estimation.

  The sound source separation method is performed by the following equation.

[Confirmation experiment]
A confirmation experiment was conducted for the purpose of evaluating the sound source separation performance of the present invention. The experimental conditions will be described. 30 sets of observation signals were prepared, and the number of sound sources in all the observation signals was N _s = 2. Each observation signal was composed of two male utterances, two female utterances, or a mixed sound of one female and one male utterance.

  The sampling frequency was 16 kHz. The two microphone signals included in each observation signal were synthesized by adding signals on the computer so that the time difference between the microphones related to each speaker's utterance was ± 1.5 milliseconds (mixing condition 1). As the initial value of the sound source model parameter, the value initialized by the above-described initialization method when there is no hidden Markov model learned in advance was used. And the prior probability density function demonstrated in the modification 1 was utilized. Each hidden Markov model has 4 states.

  The experimental results are shown in FIGS. The vertical axis in FIG. 5 represents the average power (dB) of the output distribution in each state for each of the hidden Markov models related to the sound source power feature amount estimated from the sound before mixing, and the horizontal axis represents the frequency (kHz). FIG. 5A shows the result estimated from the sound before mixing, and FIG. 5B shows the result estimated from the mixed sound.

  The parameter estimated from the mixed sound (FIG. 5B) is very similar to the parameter estimated from the sound before mixing, and proves the high reliability of the parameter estimation accuracy of the present invention.

  FIG. 6 shows the relationship between the cepstrum distortion of the signal after separation and the number of mixed sounds. The vertical axis represents the cepstrum distortion (dB), and the horizontal axis represents the number of mixed sounds (the observed signal becomes longer as the number increases). The characteristic indicated by the thick solid line (□) indicates the case where the sound source model parameter is learned from each sound signal before mixing in the present invention. The characteristic indicated by the solid line (o) indicates the case where the sound source model parameter is estimated from only the observation signal in the present invention. The characteristic indicated by the alternate long and short dash line (Δ) indicates the case where the method of Non-Patent Document 1 is used.

  6A shows the result of processing the mixed sound under the mixing condition 1, and FIGS. 6B and 6C show the mixed sound (mixing conditions 2 and 3) recorded in another reverberant environment. The result of processing is shown. In all cases, the sound source separation method of the present invention achieves sound source separation with a smaller cepstrum distortion than the method of Non-Patent Document 1. In addition, even when the sound source model parameters are not learned in advance, it can be confirmed that performance equivalent to that obtained when the learning is performed in advance is realized.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording media, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (7)

Prior probability density function of sound source power parameters representing the state of the entire sound source power time series of each of the plurality of sound source signals, and a model of sound source power features at each time frequency point of each sound source signal when the sound source power parameters are given And a sound source model storage unit storing
A feature extraction unit that extracts a sound source position feature amount and a sound source power feature amount at each time frequency point by using an observation signal obtained by converting a time domain signal obtained by collecting the plurality of sound source signals with a plurality of microphones into a time frequency domain signal. When,
The sound source power feature amount, the sound source occupancy which is a posterior probability density function of the exclusive sound source under which the observation signal is obtained, the prior probability density function of the sound source power parameter, and the sound source power of each sound source signal A sound source power parameter for updating a sound source power parameter of each sound source by inputting a feature amount model, a sound source model parameter which is a parameter for controlling the behavior of each of the sound source power feature amount model and the prior probability density function. Update section,
A sound source position parameter update unit that updates the sound source position parameter of each sound source, using the sound source position feature amount and the sound source occupancy as inputs,
The sound source model parameter is obtained by inputting the sound source power feature amount, the sound source power parameter, the sound source occupancy, the prior probability density function of the sound source power parameter stored in the sound source model storage unit, and the model of the sound source power feature amount. A sound source model parameter update unit to be updated;
The sound source position feature amount, the sound source power feature amount, the updated sound source power parameter and the sound source position parameter of each sound source, the sound source model parameter, a prior probability density function of the sound source power parameter stored in the sound source model storage unit, A sound source occupancy update unit that updates a sound source occupancy of each of the sound sources as an input of a model of a sound source power feature;
A sound source parameter estimation apparatus comprising: In the sound source parameter estimation apparatus according to claim 1,
The time series of the sound source power feature amount follows a hidden Markov model, and the posterior probability density function of the sound source power parameter under a condition where the state of the Markov model of each sound source signal is known is expressed by a covariance matrix as a diagonal matrix. A sound source parameter estimation apparatus characterized by being modeled by a Gaussian distribution and including the mean of the Gaussian distribution and a diagonal element of a covariance matrix in the sound source model parameter. The sound source parameter estimation device according to claim 1 or 2,
The sound source power feature amount output by the sound source parameter estimation device, the sound source occupancy, the sound source power feature amount, the sound source power parameter, the sound source model parameter updated by the sound source parameter estimation device, and the respective sound source signals at each time frequency point A sound source separation unit that obtains a sound source separation signal of each of a plurality of sound sources by input of a model of a sound source power feature amount by least square error estimation;
A sound source separation apparatus comprising: A feature extraction process that extracts the sound source position feature and sound source power feature at each time frequency point using the observation signal obtained by converting the time domain signal obtained by collecting multiple sound source signals with multiple microphones into the time frequency domain signal. ,
The sound source power feature amount, the sound source occupancy that is the posterior probability density function of the exclusive sound source under which the observation signal is obtained, and the prior probability density function of the sound source power parameter stored in the sound source model storage unit, The sound source power feature value model of each sound source signal, the sound source power feature value model, and a sound source model parameter that is a parameter for controlling the behavior of each of the prior probability density functions, and the sound source power of each sound source Sound source power parameter update process for updating parameters,
A sound source position parameter update process for updating the sound source position parameter of each sound source, using the sound source position feature amount and the sound source occupancy as inputs.
The sound source model parameter is obtained by inputting the sound source power feature amount, the sound source power parameter, the sound source occupancy, the prior probability density function of the sound source power parameter stored in the sound source model storage unit, and the model of the sound source power feature amount. Sound source model parameter update process to be updated,
The sound source position feature amount, the sound source power feature amount, the updated sound source power parameter and the sound source position parameter of each sound source, the sound source model parameter, a prior probability density function of the sound source power parameter stored in the sound source model storage unit, A sound source occupancy update process for updating the sound source occupancy of each of the above sound sources by inputting a model of a sound source power feature,
A sound source parameter estimation method comprising: In the sound source parameter estimation method according to claim 4,
The time series of the sound source power feature amount follows a hidden Markov model, and the posterior probability density function of the sound source power parameter under a condition where the state of the Markov model of each sound source signal is known is expressed by a covariance matrix as a diagonal matrix. A sound source parameter estimation method characterized by being modeled by a Gaussian distribution and including the mean of the Gaussian distribution and a diagonal element of a covariance matrix in the sound source model parameter. The sound source power feature amount extracted by the sound source parameter estimation method according to claim 4, the sound source occupancy, the sound source power feature amount, the sound source power parameter, the sound source model parameter updated by the sound source parameter estimation method, and each of the sound sources A sound source separation process for obtaining a sound source separation signal of each of a plurality of sound sources by input of a model of a sound source power feature amount at each time frequency point of the signal by least square error estimation;
A sound source separation method comprising:   A program for causing a computer to function as the sound source parameter estimation device or the sound source separation device according to any one of claims 1 to 3.

Images