JP2011164467A - Model estimation device, sound source separation device, and method and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound source separation device which correctly operates even if the number of sound sources is not known, and which attains sound source separation without any permutation problem between frequency components. <P>SOLUTION: A frequency domain conversion unit converts each observation signal of a time domain in each microphone to an observation signal spectrum of each frequency domain, and a phase difference calculation unit calculates a phase difference of the observation signal spectrum in each microphone. A model estimation unit sequentially applies the observation signal spectrum to a spectrum probability model for indicating distribution of a spectrum, and applies the phase difference between microphones to a phase difference probability model for indicating distribution of the phase difference, respectively, and calculates a model parameter of each probability model suitable for signal extraction and presence probability of each sound source, by using a predetermined evaluation function for evaluating each probability model. An effective sound source is extracted by using the model parameter of each probability model and the presence probability of each sound source, the observation signal spectrum is separated for each effective sound source by using a mask corresponding to the effective sound source. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention belongs to a sound source separation technique for estimating each original signal from acoustic data in which a plurality of signals are mixed, and in particular, a plurality of signals are mixed without using information on each of the original signals and how they are mixed. The present invention relates to a model estimation device, a sound source separation device, a method and a program thereof belonging to a blind sound source separation technique for estimating each original signal from only the acoustic data being processed.

  FIG. 9 shows a sound source separation device 10 configured based on a conventional blind sound source separation technique (for example, Non-Patent Document 1). At a certain time t, the signals emitted from the M sound sources and mixed with the noise are observed with the two microphones # 1 and # 2,

Suppose that

  First, in the frequency domain transform unit 110, the observation signal in the time domain is subjected to a short-time Fourier transform.

Convert to n is an index of a time frame for performing Fourier transform, and f is an index of a frequency component. Hereinafter, when there is no notice, the observation signal indicates a frequency domain signal, and in the case of a time domain observation signal, it is specified.

  Here, the observed signal spectrum is

It is assumed that Here, h _{f, L, m} is the frequency response from the sound source m (m = 1, 2,..., M) to the microphone L (L = 1, 2), and _{Sn, f, m} is the sound source m. Frequency domain representation of signal, n (= 0, ..., N _n -1) is time,

Represents frequency, _{f s} is the sampling frequency, F is the number of sampling points, a.

In order to perform sound source separation, the sound source is sparse, that is, the sound source signal s _{n, f, m} rarely takes a large value, and at most one sound source Sn _{, f, m at} each time frequency (n, f) _. Assume that only _m takes a large value. This is a property confirmed by different audio signals. Assuming this, equation (1) becomes

Can be written. Here, S _{n, f, m} is a sound source signal dominant at the time frequency (n, f).

Next, in the phase difference calculation unit 120, the phase difference between the observed signal spectra in the microphone # 1 and the microphone # 2 (referred to as the inter-microphone phase difference) _{An, f} = arg [ _{Xn, f, 1} / _{Xn, f, 2} ] is calculated. This inter-microphone phase difference _{An, f} is determined by the positional relationship between the sound source of the signal and the microphone, and if the positions of the sound sources are different from each other, _{An, f} takes a value specific to each sound source.

Next, the phase difference classification unit 31 clusters the inter-microphone phase difference _{An, f} for each frequency. From Equation (2) assuming sparseness _, the phase difference μ _{n, f, m} corresponding to the sound source m is the time frequency (n, f) where the sound source m is dominant, and the time frequency ( Since the phase difference μ _{n, f, m ′} corresponding to the sound source m _′ is obtained at n, f), when the phase difference _{An, f} is clustered, a cluster corresponding to each sound source component is formed. Here, in the conventional method, in order to specify how many clusters to create by clustering, the number of sound sources M is read from the sound source number holding unit 32, and the phase difference classification unit 31 performs clustering using the k-means method or the like. Since clustering is performed for each frequency, the correspondence between the cluster index and the sound source index corresponding to the cluster varies from frequency to frequency. For example, at a certain frequency f, the first cluster corresponds to the sound source 1 and the second cluster corresponds to the sound source 2, but at another frequency f ′, the first cluster becomes the sound source 2 and the second cluster becomes the sound source 1. In general, the correspondence between the cluster and the sound source is dispersed. This is called a permutation problem. Therefore, in order to solve this permutation problem, a permutation resolution unit 33 is provided. Here, the cluster index and the sound source index are aligned for all frequencies, and the cluster and the sound source completely correspond one to one. Arrange to do. This is performed, for example, as follows. First, for each cluster obtained at each frequency, an average value A _f of the phase differences _{An, f} within the cluster is obtained. Next, A _f / 2πf obtained by normalizing the average value A _f with the frequency f is clustered, and the frequency components corresponding to the same sound source are collected. This makes it possible to align the cluster index and the sound source index at all frequencies. Eventually, the m th cluster C _m A _n corresponding to the sound source _m, contains only the component of _f.

Next, the sound source separation unit 40 refers to C _m and takes 1 at the time frequency (n, f) forming a cluster corresponding to the sound source m, and 0 at other time frequencies (n, f). Make M _{n, f, m} . This is made for all sound sources m. Furthermore, the mask M _{n, f, m} is multiplied by one of the observation signals (here, X _{n, f, 1} ) to obtain a separated signal Y _{n, f, m} .

Y _{n, f, m} = X _{n, f, 1}・ M _{n, f, m} (3)
Finally, the time domain conversion unit 150 converts the obtained separated signal Y _{n, f, m} into a time domain signal.

H. Sawada, S. Araki and S. Makino, "A two-stage frequency-domain blind source separation method for underdetermined convolutive combination", Proc. WASPAA2007, 2007, p.139-142

As described above, the conventional method has a problem of permutation between frequencies, and it is indispensable to solve it. However, when solving, there is a problem that the clustering of A _f / 2πf, which is often used in the permutation resolution unit 33, does not work well when there is much room reverberation or when the microphone interval is wide. That is, without taking a constant value the value of A _f / 2 [pi] f is at each frequency in order to have a phase difference depends on the frequency between microphone when the reverberation of the room is large, clustering A _f / 2 [pi] f becomes difficult. When the microphone interval is wide, the actual phase difference between microphones exceeds ± 2π in the calculation of A _{n, f} = arg [x _{n, f, 1} / x _{n, f, 2} ]. for calculated in a _n of _arg, the value of _f is crowded pressing in the range of -2.pi. ≦ a _{n, f} ≦ 2 [pi, the values of a _f / 2 [pi] f is not take a constant value at each frequency, the a _f / 2 [pi] f Clustering becomes difficult. In addition, since it is necessary to know the number M of sound sources to be separated in the conventional method, it is difficult to apply when the number M of sound sources is unknown.

  An object of the present invention is to provide a model estimation device that operates even when the number of sound sources is unknown and can perform good sound source separation without causing the problem of permutation between frequency components, and a sound source separation device using the model estimation device There is to do.

  The model estimation device of the present invention is a model estimation device that observes signals from a plurality of mixed sound sources with a plurality of microphones, and extracts the signals of each mixed sound source. And a model estimation unit. The frequency domain conversion unit converts the observation signal in the time domain of each microphone into an observation signal spectrum in the frequency domain. The phase difference calculation unit calculates a phase difference (observation phase difference between microphones) between observed signal spectra in each microphone. The model estimation unit sequentially applies the observed signal spectrum to a spectrum probability model indicating a spectrum distribution and the phase difference between microphones to a phase difference probability model indicating a phase difference distribution, and evaluates each probability model. Using a predetermined evaluation function, the model parameter of each probability model suitable for signal extraction and the existence probability of each sound source are calculated.

  Further, a sound source separation device of the present invention includes the model estimation device, a signal separation unit, and a time domain conversion unit. The signal separation unit extracts an effective sound source based on the existence probability of each sound source, and creates a mask corresponding to each effective sound source using a model parameter of each probability model and a posteriori probability calculated based on the existence probability of each sound source. Then, a separated signal is generated by separating the observed signal spectrum for each effective sound source using the mask. The time domain conversion unit converts the separated signal for each effective sound source into a time domain signal.

  According to the model estimation apparatus of the present invention and the sound source separation apparatus using the model estimation apparatus, the sound source separation is performed without causing the problem of permutation between frequency components, even when the number of sound sources is unknown. be able to.

The block diagram which shows the structural example of the model estimation apparatus 100 of this invention. The figure which shows the example of a processing flow of the model estimation apparatus 100 of this invention. The figure which shows a mode that the frequency component of a signal synchronizes. The block diagram which shows the structural example of the sound source separation apparatus 200 of this invention. The figure which shows the example of a processing flow of the sound source separation apparatus 200 of this invention. The figure which shows the example of the mask obtained by the mask production | generation part 142. FIG. In FIG. 6, the figure which shows the example of the frequency characteristic of a phase difference parameter (average value) in case of m = 4, 5, and the time characteristic of a spectrum parameter. The figure which shows the performance comparison of the model estimation apparatus 200 of this invention and the conventional sound source separation apparatus 10. FIG. The block diagram which shows the structural example of the conventional sound source separation apparatus 10. FIG.

  Hereinafter, embodiments of the present invention will be described in detail.

  FIG. 1 is a block diagram showing a configuration example of the model estimation apparatus 100 of the present invention, and FIG. 2 shows a processing flow example thereof. The model estimation apparatus 100 is a model estimation apparatus that observes signals from a plurality of sound sources mixed with noise with a plurality of microphones, and extracts each of the mixed signals, and includes a frequency domain conversion unit 110 and a phase difference calculation unit. 120 and a model estimation unit 130.

  The frequency domain conversion unit 110 and the phase difference calculation unit 120 are the same as those of the conventional sound source separation device 10. In other words, it was obtained by observing a mixed signal emitted from M sound sources at a certain time t with two microphones # 1 and # 2.

By the short-time Fourier transform in the frequency domain transform unit 110

(S1). n is an index of a frame to be subjected to Fourier transform, and f is an index of a frequency component. The phase difference calculation unit 120 includes a phase difference between the observation signal spectrum of the microphone # 1 and the observation signal spectrum of the microphone # 2 (hereinafter referred to as “phase difference between microphones”) _{An, f} = arg [ _{Xn, f, 1.} / X _{n, f, 2} ] is calculated (S2).

Hereinafter, the observation signal spectrum of the microphone # 1 is denoted as X _{n, f} and used for the description.

  The model estimation unit 130 sequentially applies the phase difference between microphones to a phase difference probability model indicating a phase difference distribution and the observation signal spectrum to a spectrum probability model indicating a spectrum distribution, respectively, and evaluates each probability model. Using the evaluation function, model parameters and the like of each probability model suitable for signal extraction are calculated (S3-5).

  The phase difference probability model indicating the phase difference distribution and the spectrum probability model indicating the spectrum distribution are modeled as follows.

When the positions of the sound sources are fixed and the directions of all the sound sources viewed from the microphones are different, the inter-microphone phase difference _{An, f} takes a unique value for each sound source m. Therefore, in the present invention, the distribution of the inter-microphone phase difference _{An, f} with respect to the sound source m is modeled as a normal distribution with an average μ _{f, m} and a variance σ ² _{f, m} as follows.

This is called a phase difference probability model. The phase difference distribution is defined for each frequency f. N is a normal distribution

It is. Based on the above, the model parameters of the phase difference probability model are
θ _A = {μ _{f, m} , σ ² _{f, m} }
It can be expressed as.

In addition, in order to model the observed signal spectrum X _{n, f} , the present invention assumes the sparseness of the sound source as in the equation (2). In addition, for simplicity of description, it is assumed that the frequency response | h _{f, 1, m} | = 1 from the sound source m to the microphone 1 and arg (h _{f, 1, m} ) = 0. As a result, equation (2) becomes

It can be expressed as. Based on this assumption, the observed signal spectrum X _{n, f} is modeled as a complex normal distribution with an average value of 0 and a variance γ ² _{n, f, m} as follows.

This is called a spectral probability model. Where N _c is the complex normal distribution

It is. M is the number of mixtures. If the number of sound sources is known, the same number is used, and if the number of sound sources is unknown, a sufficiently large number (for example, M = 10) is used. The variance γ ² _{n, f, m} is an amount that represents the expected value E [| S _{n, f, m} | ² ] of the power of the sound source m. Further, by using a spectral envelope ρ _{n, m that} is time-dependent but not frequency-dependent for γ _{n, f, m} and a spectrum shape an _{n, f, m} that depends on both time and frequency, To model.

γ _{n, f, m} = a _{n, f, m}・ ρ _{n, m} (7)
Here, the spectral envelope ρ _{n, m} is the property that the onset of the signal's frequency component (the start time of the component with strong signal power) and the offset (the end point of the component with strong signal power) are synchronized at all frequencies. Is modeled. FIG. 3 shows an image of synchronization. The darker the color, the stronger the power. From this figure, it can be seen that the portions where the power of each frequency component is strong are synchronized at almost the same time. In the present invention, the spectrum shape an _{, f, m} is substituted by the amplitude | _{Xn, f} | of the observed signal spectrum. That is, a _{n, f, m} = | X _{n, f} | Based on the above, the model parameters of the spectral probability model are set to θ _X = {ρ ² _{n, m} }
It can be expressed as.

From the above, the model _{pn, f} (X _{n, f} , A _{n, f} ; θ) of the observation data (the phase difference An _{n, f} between the microphones and the observation signal spectrum X _{n, f} ) is

It becomes. Here, α _m is the existence probability p (m; θ) of the sound source m, and Σ _m α _m = 1. α _m is hereinafter referred to as a mixture weight. Further, _{pn, f} (X _{n, f} , A _{n, f} | m; θ) assumes that the inter-microphone phase difference _{An, f} and the observed signal spectrum X _{n, f} are independent from each other,

It becomes. Here, w _a and w _x are a weight for the likelihood of the phase difference and a weight for the likelihood of the spectrum, respectively.

The model estimation unit 130 uses the phase difference probability model and the spectrum probability model modeled as described above, the inter-microphone phase difference _{An, f} is used as the phase difference probability model, and the observation signal spectrum X _{n, f} is obtained. Using a predetermined evaluation function that sequentially applies each spectrum probability model and evaluates each probability model, a posteriori probability (explained later) and a parameter set suitable for signal extraction θ = {θ _A , θ _X , α _m } = {Μ _{f, m} , σ ² _{f, m} , ρ ² _{n, m} , α _m }

The model estimation unit 130 includes a posterior probability calculation unit 131, a parameter update unit 132, and a parameter holding unit 133. Prior to processing by the model estimation unit 130, an initial value θ ⁰ of the parameter set θ = {μ _{f, m} , σ ² _{f, m} , ρ ² _{n, m} , α _m } is prepared in the parameter holding unit 133. In addition, the initial value of the parameter update count index t, the mixing number M, the maximum parameter update count T, or the threshold value Δ for convergence determination is set (S0). Note that it may be performed at any time before the process in the model estimation unit 130.

The posterior probability calculation unit 131 uses the observed signal spectrum X _{n, f} , the phase difference An _{n, f} between microphones _, and the current parameter set θ ^t = {μ ^t _{f, m} , (σ ² _{f , m} ) ^t , (ρ ² _{n, m} ) ^t , α ^t _m }, the a posteriori probability pm _{n, f} , that is, the inter-microphone phase difference An _{n, f} and the observed signal spectrum X _{n, f} The probability that it is due to the signal from each sound source m at (n, f) is calculated as follows (S3).

Here, _{w a} and _{w x} is for example _w a = _1.0, the like w x = 0.2.

The parameter update unit 132 includes a spectrum parameter update unit 132a, a phase difference parameter update unit 132b, and a mixture weight update unit 132c, and updates the current parameter set θ ^t to θ ^{t + 1} (S4).

The spectrum parameter updating unit 132a updates the model parameter (ρ ² _{n, m} ) ^t of the spectrum probability model using the posterior probability pm _{n, f} by the following calculation (S4-1).

Here, N _f is the number of frequency components.

The phase difference parameter updating means 132b uses the posterior probability pm _{n, f} and the inter-microphone phase difference _{An, f} to model parameter θ _A ^t = {μ ^t _{f, m} , (σ ² _{f, m} ) ^t } is updated by the following calculation (S4-2).

The mixture weight calculation unit 132c updates the mixture weight α ^t _m by the following calculation using the posterior probability pm _{n, f} (S4-3).

Here, N _n is the number of time frames.

  The basis for deriving each update formula (11) to (14) in the parameter update unit 132 will be described. The parameter update is performed based on the EM algorithm. The normal distribution index m is treated as a hidden variable in the EM algorithm. First, the cost function L (θ) for maximum likelihood estimation is given as follows.

Here, p (m | θ) is the mixture weight α _m , and _{pn, f} (X _{n, f} , A _{n, f} | m; θ) is as shown in Equation (9).
W _a and w _x are weights for the likelihood of the phase difference and the likelihood of the spectrum, respectively. The evaluation function (Q function) used in the EM algorithm is given as follows.

This Q function gives a higher evaluation value as the spectral envelope in which the onset and the offset are synchronized is clustered into one cluster. That is, for each signal, an evaluation is given that it is more suitable for signal extraction as the strength of each frequency component is more synchronized.
The updated parameter set θ ^{t + 1} = {μ ^{t + 1} _{f, m} , (σ ² _{f, m} ) ^{t + 1} , (ρ ² _{n, m} ) ^{t + 1} , α ^{t + 1} _m } Estimated to maximize the Q function. That is, the equation (11) for obtaining the model parameter (ρ ² _{n, m} ) ^{t + 1} of the spectral probability model is

Equations (12) and (13) for obtaining model parameters μ ^{t + 1} _{f, m} and (σ ² _{f, m} ) ^{t + 1} of the phase difference probability model

Equation (14) for obtaining the mixture weight α ^{t + 1} _m derived from

Is derived by

Parameter holding unit 133 stores the parameter set theta ^{t + 1} obtained by the update processing in the parameter update unit 132, a parameter set theta ^t at the next process in the posterior probability estimation unit 131 and the parameter updating unit 132 As offered.

In the model estimation unit 130, the posterior probability calculation unit 131 and the parameter update unit 132 (and the read / write of update data to the parameter holding unit 133) reach the maximum parameter update count T set in advance, or each parameter value This is repeated until the fluctuation range due to updating becomes smaller than the threshold value Δ for convergence determination. The model estimation unit 130 then sets the parameter set θ ^e = {μ ^e _{f, m} , (σ ^e _{f, m} ) ² , (ρ ^e _{n, m} ) ² , α ^e _m } after the iteration and at that time. The posterior probability pm ^e _{n, f of} is output.

  The sound source separation device 200 can be configured by adding a signal separation unit 140 and a time domain conversion unit 150 to the model estimation device 100 described in the first embodiment as illustrated in FIG. The processing flow is shown in FIG.

The signal separation unit 140 includes an effective sound source estimation unit 141, a mask creation unit 142, and a separated signal creation unit 143, and separates the signal of each sound source from the observed signal spectrum X _{n, f} (S6).

The effective sound source estimation unit 141 extracts an index of a sound source that actually exists (hereinafter, referred to as “effective sound source”) from the M indexes of the number of mixtures M used in the calculation. Specifically, when the number of sound sources is known and the number of mixing M = the number of sound sources, all indexes m are output. If the number of sound sources is unknown, m that satisfies a sufficiently large value (for example, α ^e _m > ε (ε is 10 ⁻⁶ or the like)) among the updated mixture weight α ^e _m is determined as an effective sound source, Output all m.

The mask creation unit 142 creates a mask M _{n, f, m} for extracting each sound source corresponding to the index m of the sound source output as an effective sound source. The mask M _{n, f, m} uses the updated posterior probability pm ^e _{n, f} ,
M _{n, f, m} = pm ^e _{n, f} (17)
It can ask for.

Separation signal generator 143 multiplies the mask M _{n, f,} a _m observed signal spectrum X _n, the _f, to calculate the separation signal Y _{n, f, m.}

Y _{n, f, m} = X _{n, f} · M _{n, f, m} (18)
Finally, the time domain conversion unit 150 converts the separated signal Y _{n, f, m} into a time domain signal y _m (t) for each sound source m and outputs it.

  As described above, the model estimation apparatus 100 and the sound source separation apparatus 200 described in the first and second embodiments can extract an effective sound source even when the number of sound sources is unknown, and cause a problem of permutation between frequency components. The sound source can be separated well without any problems. The reason will be explained.

-Reason why effective sound source can be extracted Equation (6), which represents a spectrum model, shows that the likelihood is larger when a small number of clusters have a large variance. That is, Equation (6) has the effect of explaining the observation signal with as few clusters as possible. As a result, only the mixing weight α _m corresponding to the index m corresponding to the effective sound source has a large value, and the mixing weight (α _m ′) corresponding to the other index _m ′ is infinitely close to 0. An effective sound source can be extracted.

Reason why permutation problem does not occur Maximization of the first term of the evaluation function (16) can be interpreted as separation by phase difference clustering at each frequency. It can be interpreted as clustering of spectral envelopes with synchronized offsets. That is, Equation (16) has a configuration in which separation can be performed with the first term while essentially preventing the problem of permutation for each frequency by maximizing the second term.

In each of the above embodiments, the spectrum shape an _{, f, m} is substituted with the amplitude | _{Xn, f} | of the observed signal spectrum, but the spectrum shape is included in the model parameter θ as a parameter af _{, m} that does not depend on time. The spectrum parameter updating unit 132a may calculate the value. In this case, the spectrum parameter updating unit 132a calculates the following equations (19) to (21).

Here, equation (20) are those given a _f, in order to eliminate the scaling ambiguity _m and [rho _{n, m,} a Σ _f a _{f, m} = 1 constraint.

In each of the embodiments described above, when there are two microphones, that is, as the phase difference between microphones, the phase difference _{An, f} = arg [ _{Xn, f, 1} / _{Xn, f, 2} ] is used, but two or more microphones can be used. That is, consider a vertical vector in which the phase difference A _{jj′n, f} = arg [X _{n, f, j} / X _{n, f, j ′} ] of observation signals between microphones #j and _{j ′} is arranged for all microphone pairs. Thus, the phase difference between microphones can be modeled. In this case, the expression (4) is expanded to a plurality of microphones, and the distribution of the phase difference between the microphones related to the sound source m is

Model with. At this time, the phase difference parameter updating unit 132b

Calculate

<Effect of the invention>
In order to confirm the effect of the present invention, a sound source separation experiment was performed using the conventional method and the method of the present invention. The number of sound sources and the number of microphones were both 2. The sampling frequency is 8 kHz, and the microphone intervals are 4 cm and 20 cm. In the invention method, the mixing number M was set to 8. On the other hand, clustering of the phase difference between microphones was performed using the k-means method as a conventional method. The number of sound sources (= clustering number) given by the k-means method was set to k = 8 similarly to the number of mixtures in the inventive method.

6, the mask M _n obtained when using the method of assuming the present invention a mixed number _{M = 8, f, m =} pm e n, the _f plots for each case of m = 1 to 8 is there. From FIG. 6, it can be seen that the mask for two signals has a large power in the method of the present invention. This result and equation (14) show that an effective sound source can be extracted.

FIG. 7 shows the frequency characteristics of μ _{f, m} (FIG. 7A) and the model parameters of the spectral probability model among the model parameters of the obtained phase difference probability model for m = 4 and m = 5 in FIG. The time characteristic of ρ _{n, m} (FIG. 7 (b)) is shown. FIG. 7A shows that a parameter μ _{f, m} having a linear phase characteristic is obtained. Further, FIG. 7 (b) shows that the spectral envelope of the signal is obtained by the spectral parameter ρ _{n, m} .

  FIG. 8 shows the evaluation of the sound source separation performance (Signal to interference ratio (SIR) and Signal to distortion ratio (SDR)) of 20 kinds of voice combinations, and the average was obtained. Is. In FIG. 8, k-means indicates the performance of the conventional method, and proposed indicates the performance of the method of the present invention. It can be seen that the method of the present invention provides higher separation performance than the conventional method.

  When the above model estimation device and sound source separation device are realized by a computer, the processing functions performed by the assignment control unit are described by a program. By executing this program on the personal computer or portable terminal through the exchange of data between the input means and various storage means and the CPU, the hardware and software cooperate to realize the above processing functions on the computer. The effects of the model estimation device and the sound source separation device of the invention are exhibited. In this case, at least a part of the processing function may be realized by hardware. Further, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

Claims (9)

A model estimation device for observing signals from a plurality of mixed sound sources with a plurality of microphones and extracting each mixed signal,
A frequency domain conversion unit that converts the observation signal in the time domain of each microphone into an observation signal spectrum in the frequency domain;
A phase difference calculation unit for calculating a phase difference between observation signal spectra in each microphone (hereinafter referred to as “phase difference between microphones”);
A predetermined evaluation function for evaluating each probability model is obtained by sequentially applying the observed signal spectrum to a spectrum probability model indicating a spectrum distribution and the phase difference between microphones to a phase difference probability model indicating a phase difference distribution. A model estimator that calculates model parameters for each probability model suitable for signal extraction,
A model estimation device comprising: The model estimation apparatus according to claim 1,
The model estimation apparatus according to claim 1, wherein the evaluation function gives an evaluation that each signal is more suitable for signal extraction as the strength of each frequency component is more synchronized. The model estimation apparatus according to claim 1 or 2,
The model estimation unit includes:
The observed signal spectrum, the phase difference between the microphones, the model parameter of the phase difference probability model, the model parameter of the spectrum probability model, and the existence probability of each sound source (hereinafter referred to as “mixing weight”) stored in the parameter holding unit )), The posterior probability calculation unit for calculating the probability that the observed signal spectrum and the phase difference between the microphones are due to signals from each sound source at each time frequency (hereinafter referred to as “posterior probability”);
Spectral parameter updating means for updating the model parameter of the spectral probability model using the posterior probability, Phase difference parameter updating means for updating the model parameter of the phase difference probability model using the posterior probability, A parameter updating unit comprising: a mixing weight updating unit configured to update the mixing weight using a posterior probability;
A parameter holding unit for storing each model parameter and mixing weight, which is updated by the parameter updating unit;
A model estimation apparatus comprising: The model estimation device according to any one of claims 1 to 3,
An effective sound source is extracted based on the updated mixed weight, a mask is created using the updated posterior probability corresponding to each effective sound source, and the observed signal spectrum is determined for each effective sound source using the mask. A signal separation unit for generating a separated separated signal;
A time domain converter that converts the separated signal for each effective sound source into a signal in the time domain;
A sound source separation device comprising: A model estimation method for observing signals from a plurality of mixed sound sources with a plurality of microphones and extracting each mixed signal,
A frequency domain conversion step for converting an observation signal in the time domain in each microphone into an observation signal spectrum in the frequency domain, and
A phase difference calculating step for calculating a phase difference between observed signal spectra in each microphone;
The observed signal spectrum is sequentially applied to a spectrum probability model indicating a spectrum distribution, and the phase difference between the observed signal spectra is sequentially applied to a phase difference probability model indicating a phase difference distribution, and each probability model is evaluated. A model estimation step for calculating a model parameter of each probability model suitable for signal extraction using an evaluation function;
A model estimation method comprising: The model estimation method according to claim 5, comprising:
The model estimation method according to claim 1, wherein the evaluation function gives an evaluation that each signal is more suitable for signal extraction as the strength of each frequency component is more synchronized. A model estimation method according to claim 5 or 6,
The model estimation step includes:
The phase difference between the observed signal spectrum and the observed signal spectrum, the model parameter of the phase difference probability model, the model parameter of the spectral probability model, and the existence probability of each sound source (hereinafter referred to as “mixed”) The probability (hereinafter referred to as “posterior probability”) that the observed signal spectrum and the phase difference between the observed signal spectra are due to the signal from each sound source at each time frequency is calculated. A posteriori probability calculation step;
A spectral parameter update substep for updating the model parameter of the spectral probability model using the posterior probability, and a phase difference parameter update substep for updating the model parameter of the phase difference probability model using the posterior probability; A parameter update step for performing a mixture weight update substep for updating the mixture weight using the posterior probability; and
A parameter holding step for storing each model parameter and the mixing weight updated in the parameter updating step in a parameter holding unit;
Is repeatedly executed a predetermined number of times or until the values of the respective model parameters and the mixture weights converge. A model estimation method according to any one of claims 5 to 7,
An effective sound source is extracted based on the updated mixed weight, a mask is created using the updated posterior probability corresponding to each effective sound source, and the observed signal spectrum is determined for each effective sound source using the mask. A signal separation step for generating a separated separated signal;
A time domain conversion step of converting the separated signal for each effective sound source into a time domain signal;
Sound source separation method to perform.   A program for causing a computer to execute the method according to claim 5.