JP5881454B2 - Apparatus and method for estimating spectral shape feature quantity of signal for each sound source, apparatus, method and program for estimating spectral feature quantity of target signal - Google Patents

Description

  The present invention relates to a technique for estimating a spectral shape feature quantity of a signal for each sound source and a technique for estimating a spectral feature quantity of a target signal from observation signals collected by a microphone by mixing acoustic signals emitted from a plurality of sound sources. .

FIG. 1 shows a functional configuration example of a conventional target signal spectrum feature quantity estimation device disclosed in Non-Patent Documents 1, 2, and 3. This target signal spectrum feature quantity estimation apparatus 900 has a discrete state number model storage unit associated with a sound source for each sound source (hereinafter, distinguished by numbers m = 1, 2,...) Included in the observed signal. 901-m, a state-dependent spectrum model storage unit 902-m, and a discrete state number selection unit 903-m, and further include a feature extraction unit 904, a sound source occupancy update unit 905, and a target sound spectrum estimation unit 906. Yes. In FIG. 1, the case where the number of sound sources is two is illustrated for simplicity, but generally three or more sound sources can also be considered. In this case, the discrete state number model storage unit 901-m and the state-dependent spectrum are used. The model storage unit 902-m and the spectrum number selection unit 903-m are prepared as many as the number of sound sources.

In the following description, it is assumed that the number of sound sources is N (m). A sound source m = 1 is a target sound source, and a sound source m> 1 is a background sound source. n is time, k (= 1,2, ..., N (k)) is frequency, and a variable that takes a value at a certain time frequency point (n, k) like x _{n, k} X _n (= [x _{n, 1} , ..., x _{n, N (k)} ] ^T ) when referring to a vector (for example, a spectral feature, etc.) that can be expressed and combined at all frequencies It shall be written as T represents a non-conjugated transpose of a matrix or vector.

The discrete state number model storage unit 901-m stores a prior probability function p (q ^(m) ) related to the discrete state number q ^(m) representing the discrete state of the spectral feature quantity for the short time frame for the sound source m. . The discrete state number q ^(m) is assumed to be one of N _q ^(m) (> 0) numbers (natural values from 1 to N _q ^(m)) .

The state-dependent spectrum model storage unit 902-m has a conditional probability density function p (s ^{(m) for} the spectral feature s ^(m) of the sound source m when the discrete state number q ^(m) is given for the sound source m. | q ^(m) ) is stored.

The feature extraction unit 904 receives the time domain signal x (t) collected by the microphone and extracts the spectral feature amount x _n of the observation signal for each short-time frame n.

The discrete state number selection unit 903-m includes the spectral feature quantity x _n of the observed signal, the prior probability function p (q ^(m) ) of the discrete state number relating to the sound source m, and the conditional probability density function p (s ^{( m)} | q ^(m) ), estimated sound source occupancy M ^ _n ^(m) (= [M ^ _{n, 1} ^(m) , ..., M ^ _{n, N (k)} ^(m) ] ^T ) And the estimated value q ^ _n ^(m) of the discrete state number of the sound source m that best matches the observed sound.

The sound source occupancy update unit 905 includes the spectral feature amount x _n of the observed sound, the conditional probability density function p (s ^(m) | q ^(m) ) of the spectral feature amount for all sound sources m, and all sound sources m. Receive a combination of estimated discrete state numbers {q ^ _n ⁽¹⁾ , ..., q ^ _n ^{(N (m))} } for each of the sound sources, and each sound source m is the largest at each time frequency point The estimated value M ^ _{n, k} ^(m) of the sound source occupancy, which is the posterior probability with energy, is _obtained .

Further, the target sound spectrum estimation unit 906 outputs an estimated value M ^ _n ⁽¹⁾ of the sound source occupancy of the target sound, an estimated value q ^ _n ⁽¹⁾ of the discrete state number of the target sound, and the feature extraction unit 904 outputs. The estimated value of the spectrum of the target sound is obtained using the spectral feature quantity x _n of the observed signal and the conditional probability density function p (s ⁽¹⁾ | q ⁽¹⁾ ) of the spectral characteristic quantity of the target sound as inputs.

In the conventional target signal spectrum feature amount estimation apparatus 900, each of the sound source occupancy update unit 905 and each discrete state number selection unit 903-m efficiently uses the spectrum number that best matches the sound source occupancy and the spectrum feature amount of the observation signal. The following two assumptions were made in order to be sought after.
<1>
At each time frequency point (n, k), the spectral feature value x _{n, k} of the observed signal matches the larger one of the spectral feature values sn _{, k} ^(m) of each sound source m. That is,
x _{n, k} = max {s _{n, k} ⁽¹⁾ , ..., s _{n, k} ^{(N (m))} } (1)
<2>
The spectral feature value s _n ^(m) of each sound source signal is independent for each frequency, given the discrete state number q (m). That is, the conditional probability density function p (s _n ^(m) | q ^(m) ) of the spectral feature quantity can be decomposed as follows. However, s _n ^(m) = [s _{n, 1} ^(m) ,..., Sn _{, N (k)} ^(m) ] ^T.

  As disclosed in Non-Patent Documents 1, 2, and 3, under the above two assumptions, the discrete state number and the sound source occupancy are updated by alternately and repeatedly updating the discrete state number and the sound source occupancy of each sound source. Can be estimated. At this time, in each repetition, the discrete state number can be updated separately (independently) for each sound source, and the sound source occupancy can be updated separately (independently) for each frequency. That is, in each update, since there is no need to consider a combination between sound sources and a combination between frequencies, each update can be performed with a low calculation cost.

Tomohiro Nakatani, Akiko Araki, Takuya Yoshioka, Masayoshi Fujimoto, “Sound Source Separation Based on DOA Clustering and Logarithmic Spectrum HMM of Speech,” Proc. Year. Tomohiro Nakatani, Akiko Araki, Takuya Yoshioka, Masayoshi Fujimoto, “Multi-channel sound source separation based on unsupervised simultaneous learning of sound source spectrum HMM and sound source direction model,” Proc. Of the 2011 Spring Conference of Acoustical Society of Japan, pp.805- 808, March, 2011. Tomohiro Nakatani, Akiko Araki, Mark Delcroa, Takuya Yoshioka, Masayoshi Fujimoto, “Integrated speech recognition approach robust to nonstationary noise: Statistical model-based speech enhancement based on sound source direction GMM and logarithmic spectrum GMM,” Acoustical Society of Japan 2011 Proceedings of the Autumn Research Presentation, September, 2011.

  The conventional target signal spectrum feature quantity estimation apparatus has a problem that the calculation cost for obtaining these values becomes enormous if the assumption <2> is not satisfied. Therefore, in order to estimate the spectrum of the target signal with low calculation cost, it is necessary to follow the above assumption. For this reason, a mixed Gaussian distribution (or a hidden Markov model that also models the time transition of the discrete state) expressing the distribution of the logarithmic power spectrum as a discrete state model is used as the spectrum model of each signal. In order to satisfy the assumption of <2>, the covariance matrix of each Gaussian distribution constituting the mixed Gaussian distribution (or Hidden Markov Model) has to be limited to only a diagonal matrix.

  However, this limitation limits the distribution of the signal that can be expressed by the spectrum model, and does not always represent the distribution inherent in the signal with high accuracy. As a result, there is a problem that the estimation accuracy of the spectrum of the target sound and the background sound cannot be made higher than the accuracy of the spectrum that can be expressed by the limited distribution. For example, more specifically, a mixed Gaussian distribution (or a hidden Markov model for MFCC) with respect to a mel frequency cepstrum coefficient (hereinafter referred to as MFCC) often used in speech recognition is currently the best model for the spectral shape of a speech signal. However, in the conventional method, this cannot be used as a spectrum model of a signal. The reason for this is that in this model, each Gaussian distribution corresponding to each discrete state defines the distribution of MFCC, and each coefficient of MFCC is related to the entire spectral shape spanning between frequencies. This is because the spectrum distribution of a signal corresponding to a discrete state cannot generally be independent between frequencies.

  In view of such a problem, the present invention differs from a restrictive model in the prior art in that a more general model (a highly accurate model in which spectrum values have a correlation between frequencies) without restriction. Even if it is used, the spectral shape feature of each signal is estimated from the observation signal collected by the microphone mixed with acoustic signals from multiple sound sources so that the target signal spectrum can be estimated efficiently. It is an object of the present invention to provide a technique and a technique for estimating a spectral feature amount of a target signal.

Spectral shape feature amount estimation of the present invention is a vector having continuous scalar values as elements from observation signals collected by mixing acoustic signals from a plurality of sound sources , and the shape of the spectral feature amount of each acoustic signal. The spectral shape feature quantity to be represented is estimated, and the prior probability density function (spectral shape model) of the spectral shape feature quantity corresponding to each sound source and the condition of the spectral feature quantity given the spectral shape feature quantity The attached probability density function (spectrum observation model) is used. Specifically, the optimization function having the exclusive sound source number representing the sound source of the acoustic signal having the maximum energy at each time frequency point as a latent variable is maximized using the spectrum shape model and the spectrum observation model, The posterior probability (sound source occupancy) that each sound source is a sound source represented by an occupying sound source number is estimated based on the spectral shape feature amount for each sound source and the spectral shape feature amounts of all sound sources. The spectral shape model is a probability density function having a correlation between frequencies. The optimization function is the product of the conditional probability density function of the spectral features of the observed signal given the spectral shape features of all sound sources and the prior probability density function of the spectral shape features determined for each sound source. It is represented by

  The target signal spectral feature amount estimation technique of the present invention is a spectral shape feature corresponding to the sound source of the target signal out of the spectral shape feature quantity and the sound source occupancy for each sound source estimated by the spectral shape feature amount estimation technique of the present invention. The spectral feature amount of the target signal is estimated from the amount, the sound source occupancy of the target signal source, the spectrum observation model corresponding to the target signal source, and the spectral feature amount of the observation signal.

  According to the present invention, details will be given in the description of the embodiment, but as a spectrum shape model, a highly accurate model in which spectrum values have a correlation between frequencies can be introduced. Then, the spectrum of the target signal can be estimated with lower calculation cost and with higher accuracy than the conventional example.

The figure which shows the function structural example of the target signal spectrum feature-value estimation apparatus of a prior art example. The figure which shows the function structural example of the target signal spectrum feature-value estimation apparatus. The figure which shows the function structural example of the target signal spectrum feature-value estimation apparatus.

<Outline of this technology>
First, an outline of the present technology will be described, and then details of the present technology will be described.
FIG. 2 shows a configuration example of the spectrum shape feature quantity estimation device / target signal spectrum feature quantity estimation device. The target signal spectral feature amount estimation apparatus 100 includes, for each m-th sound source among a plurality of sound sources included in the observation signal, a spectrum shape model storage unit 101-m associated with the sound source and a spectrum observation model storage unit 102. -M and a spectrum shape estimation unit 103-m, and further include a feature extraction unit 104, a sound source occupancy update unit 105, and a target sound spectrum estimation unit 106. The spectrum shape feature quantity estimation device 100p differs from the target signal spectrum feature quantity estimation device 100 in that the target sound spectrum estimation unit 106 is not provided. In FIG. 2, the case where the number of sound sources is two is illustrated for simplicity, but when three or more sound sources are considered, the spectrum shape model storage unit 101-m, the spectrum observation model storage unit 102-m, and the spectrum It is assumed that different shape estimation units 103-m are prepared for the number of sound sources.

The spectral shape feature quantity estimation apparatus in the embodiment is an apparatus that estimates the spectral feature quantity of the target signal using the obtained maximum likelihood spectral shape feature quantity or the like (the target signal in the embodiment) rather than being independently present alone. In some cases, it may be practical to exist as a constituent element of a spectral feature amount estimation apparatus. Furthermore, the spectral shape feature quantity estimation device is not a component constituting the target signal spectrum feature quantity estimation device so that it can be easily separated from the target signal spectrum feature quantity estimation device. It can also be said that the evaluation was made on one side focusing on a certain function. In short, it can be said that the spectrum shape feature quantity estimation device is almost practical to be the target signal spectrum feature quantity estimation device itself.
However, the spectrum shape feature quantity estimation device exists as a single independent component, and that the target signal spectrum feature quantity estimation device is a component that constitutes the target signal spectrum feature quantity estimation device so that it can be easily separated from the target signal spectrum feature quantity estimation device. It is not intended to be excluded. For example, if the objective is to estimate the spectral shape feature quantity of each sound source, there is no obstacle to realizing the spectral shape feature quantity estimation device as a single independent component.
Here, it is assumed that the target signal spectral feature quantity estimation device / spectral shape feature quantity estimation device is realized by a computer such as a dedicated machine configured by dedicated hardware or a general-purpose machine such as a personal computer. The estimation device will be described as a component constituting the target signal spectrum feature quantity estimation device.
A hardware configuration example in the case where the target signal spectral feature quantity estimation device / spectral shape feature quantity estimation device is realized as a single component by a computer will be described later.

In the following description, it is assumed that the number of sound sources is N (m). A sound source m = 1 is a target sound source, and a sound source m> 1 is a background sound source. n is time, k (= 1,2, ..., N (k)) is frequency, and a variable that takes a value at a certain time frequency point (n, k) like x _{n, k} X _n (= [x _{n, 1} , ..., x _{n, N (k)} ] ^T ) when referring to a vector (for example, a spectral feature, etc.) that can be expressed and combined at all frequencies It shall be written as T represents a non-conjugated transpose of a matrix or vector.

The spectrum shape model storage unit 101-m stores a prior probability density function p (c ^(m) ) of the spectrum shape feature value c ^(m) for the short time frame for the sound source m. This prior probability density function is hereinafter referred to as a spectral shape model.

The spectrum observation model storage unit 102-m has a conditional probability density function p (s ^(m) | c ^{( )} of the spectrum feature s ^(m) when the spectrum shape feature c ^(m) is obtained for the sound source m. ^m) Remember). Since the spectral feature quantity s ^(m) cannot be uniquely determined corresponding to the given spectral shape feature quantity c ^(m) , it is assumed that stochastic fluctuation is included. A model of this fluctuation is a conditional probability density function p (s ^(m) | c ^(m) ) of the spectral feature s ^(m) . Hereinafter, this function is referred to as a spectrum observation model.

The feature extraction unit 104 receives the time domain signal x (t) picked up by the microphone and extracts the spectral feature amount x _n of the observation signal in each short time frame n.

The spectrum shape estimation unit 103-m is configured to observe the spectrum feature amount x _n of the observation signal, the sound source occupancy M ^ _n ^{(m) of} the sound source m, the spectrum shape model p (c ^(m) ) for the sound source m, and the spectrum observation of the sound source m. The model p (s ^(m) | c ^(m) ) is received, and the spectral shape feature c ^ _n ^(m) of the sound source m is estimated.

The sound source occupancy update unit 105 receives the spectral feature amount x _n of the observation signal from the feature extraction unit 104, and from each of all the spectral shape estimation units 103-m, the spectral shape feature amount c ^ _n ^{(m ) And} the spectrum observation model p (s ^(m) | c ^(m) ) of each sound source m is received from all spectrum observation models, and the estimated value M ^ _n ^(m) of the sound source occupancy of each sound source m is estimated To do.

The target sound spectrum estimation unit 106 receives the spectral feature amount x _n of the observed signal from the feature extraction unit 104, and receives the estimated value c ^ _n ⁽¹⁾ of the spectral shape feature amount of the target signal from the spectral shape estimation unit 103-1. Then, the estimated value M ^ _n ⁽¹⁾ of the sound source occupancy of the target signal is received from the sound source occupancy update unit 105, and the conditional probability density function p (s ⁾ of the spectral feature quantity of the target signal is received from the spectrum observation model storage unit 102-1. ⁽¹⁾ | c ⁽¹⁾ ) is received and the estimated value of the spectrum of the target signal is obtained.

Furthermore, in the present technology, in order to efficiently obtain the spectral shape feature amount of each sound source m, the following assumptions are made.
<1>
At each time frequency point (n, k), the spectral feature amount x _{n, k} of the observed signal matches the larger one of the spectral feature amounts sn _{, k} ^(m) of the sound source m. That is, Formula (3) is materialized. Hereinafter, this assumption is referred to as a LogMax model.
x _{n, k} = max {s _{n, k} ⁽¹⁾ , ..., s _{n, k} ^{(N (m))} } (3)
<2>
The spectral feature value s _n ^(m) of each sound source signal is independent for each frequency, given the spectral shape feature value c _n ^{(m) of} each sound source. That is, the conditional probability density function p (s _n ^(m) | c _n ^(m) ) of the spectral feature can be decomposed as follows. However, s _n ^(m) = [s _{n, 1} ^(m) ,..., Sn _{, N (k)} ^(m) ] ^T.

  Of these assumptions, Assumption <1> is the same as the conventional example, and Assumption <2> is different from the conventional example.

  In the above configuration, the most important difference from the conventional example is that in the conventional example, the model of the spectral feature amount of each sound source was expressed by two combinations of a discrete state model and a state-dependent spectral model. On the other hand, the present invention is expressed by two combinations of a spectrum shape model and a spectrum observation model.

  In the conventional example, the feature quantity related to the spectrum shape is expressed by a state-dependent spectrum model mixed with the spectrum observation process, and in the model, it is necessary to assume that the spectrum value is independent between frequencies.

  On the other hand, in the present invention, it is only the spectrum observation model that needs to be assumed to be independent between frequencies, and nothing is defined for the spectrum shape model. As a result, it is possible to introduce a spectrum shape model in which spectrum values have a correlation between frequencies. For example, a highly accurate spectral shape model such as a mixed Gaussian model of MFCC can be used.

  Under the above two assumptions, the spectral shape feature amount and the sound source occupancy can be estimated by alternately and repeatedly updating the spectral shape feature amount and the sound source occupancy degree of each sound source. At this time, in each iteration, the spectral shape feature amount can be updated separately for each sound source, and the update of the sound source occupancy can be updated separately for each frequency. That is, in each update, it is not necessary to consider a combination of values between sound sources and a combination of values between frequencies, so that each update can be performed with low calculation cost.

《Details of this technology》
c _n = H a (s _n), and feature transformation function for extracting a spectral shape feature c _n from the spectral characteristic quantity s _n of the signal. Here, n represents the short frame number, and the spectral feature value s _n is a vector s _n = [s _{n, 1} , s _{n, 2} , with elements of continuous scalar values s _{n, k} corresponding to the respective frequencies. ..., s _{n, N (k)} ] ^T, and the spectral shape feature value c _n is a vector c _n = [c _n having continuous scalar values c _{n, h} corresponding to each dimension of the shape parameter as elements. _{, 1} , c _{n, 2} ,..., C _{n, N (h)} ] ^T, and H (•) is an arbitrary function that extracts a feature quantity related to the shape of the spectrum. For example, the output of the often logarithmic mel filter bank used a s _n in speech recognition, when the c _n as its MFCC, H (s _n) corresponds to a discrete cosine transform D of s _n (s _n). That is, H (s _n ) = D (s _n ). Also, the s _n a log power spectrum, when the corresponding MFCC the c _n, H (s _n) is the inverse transform of the logarithmic function in s _n, i.e. after the application of the exponential function (exp (·) hereinafter) This corresponds to applying Mel filter bank processing (denoted as mfb (•)) and applying logarithmic transformation (log (•)) again. That is, H (s _n ) = log (mfb (exp (s _n ))).

  In the following discussion, basically, it is assumed that the feature extraction unit 104 extracts a plurality of short-time frames from the observation signal. However, in the following description (for example, Embodiments 1 and 2), the discussion is simplified. Therefore, the process is described as closed in one short frame n. For a plurality of short time frames, the same processing is individually applied to each short time frame. In other embodiments, this is not necessarily the case.

[Definition of optimization function]
Now, _let s _n ^{(m) be} the spectral feature quantity in the short-time frame n of the sound source m. In addition, the spectral feature values and spectral shape feature values for all the sound sources m are collectively described as follows. Here, N (m) represents the number of sound sources.
S _n = {s _n ⁽¹⁾ , ..., s _n ^{(N (m))} } (5)
S _{n, k} = {s _{n, k} ⁽¹⁾ , ..., s _{n, k} ^{(N (m))} } (6)
C _n = {c _n ⁽¹⁾ , ..., c _n ^{(N (m))} } where c _n ^(m) = H (s _n ^(m) ) (7)

In the present technology, as a step for estimating the target sound spectrum, the spectrum shape feature amount of each sound source m is obtained by posterior probability maximization (MAP) estimation as defined below.

  As will be described later, this MAP estimation is realized mainly by an iterative process by the spectrum shape estimation unit 103-m and the sound source occupancy estimation unit 105.

In order to perform MAP estimation, in the present technology, the spectral shape model of the sound source m (that is, the prior probability density function of the spectral shape feature amount) p (c _n ^(m) ) and the spectrum observation model (that is, the spectral shape feature). The conditional probability density function (p (s _n ^(m) | c _n ^(m) )) of the spectral feature is given in advance (or the target sound spectrum as described later) Can be estimated from the observed signal to be estimated).

Since no particular assumption is made regarding the spectrum shape model p (c _n ^(m) ), any model that represents the probability density function of c _n ^(m) can be used as a model. Typical examples include a normal distribution for c _n ^(m) , a Gaussian mixture model, a hidden Markov model, and the like.

The spectrum observation model p (s _n ^(m) | c _n ^(m) ) is defined as an inverse process of the feature quantity conversion H (s) described above. However, in general, c _n = H (s _n ) is a many-to-one transformation, and the inverse transformation is not uniquely determined. Therefore, the method of determination is arbitrary. Here is an example. Now, let s = G (c) be a function that transforms a representative point in the set of s that satisfies c = H (s), and call this pseudo inverse transformation of H (s). Further, e = sG (H (s)) is called an inverse conversion error. Assuming that e follows the normal distribution of expected value 0 and covariance matrix Ξ, the spectrum observation model can be defined as follows. Here, Nd (x; μ, Σ) represents a probability density function of a normal distribution of mean μ and covariance matrix Σ.
p (s _n ^(m) | c _n ^(m) ) = Nd (s _n ^(m) ; G (s _n ^(m) ), Ξ) (9)

Further, it is assumed that this spectrum observation model p (s _n ^(m) | c _n ^(m) ) can be decomposed into products of conditional probability density functions defined at respective frequencies based on Expression (4). This means that in the case of the above model, the covariance matrix is defined by the diagonal matrix Ξ = diag {ξ _k }. If the function that gives the kth element of s = [s ₁ , s ₂ ,..., s _{N (k)} ] where s = G (c) is expressed as s _k = G _k (c), (9) can be disassembled as follows.

  In the above, it is assumed that the expected value of e is E {e} = 0 (where E {·} is a function that takes the expected value of the random variable). On the other hand, even if this value is not 0, G ′ (c) with G ′ (c) = G (c) −E {e} is set to pseudo inverse transformation, so that the expected value of the inverse transformation error is 0. Modeling is possible. Furthermore, in the above description, it is assumed that the spectrum observation model (that is, the inverse transformation error) follows a normal distribution. However, the same definition is possible using not only the normal distribution but also a general distribution.

Next, in order to treat the assumption of Expression (3) introduced in the present invention stochastically, the following relational expression is introduced as in the conventional example.

Here, d _n = [d _{n, 1} ,..., D _{n, N (k)} ] represents an exclusive sound source number indicating the number of the sound source having the largest energy at each time frequency. For example, d _{n, k} = 1 indicates that the sound source of m = 1 (that is, the target sound) is the most occupied sound source at the time frequency point (n, k). δ (·) is a Dirac delta function. Equation (11) indicates that the spectral feature quantity x _{n, k} of the observed signal and the spectral feature quantity sn _{, k} ^(m) [m = d _{n, k} ] of the sound source indicated by the occupied sound source number match. In other words, equation (12) means that the exclusive sound source number matches the number of the sound source having the largest value of the spectral feature amount. This is clearly equivalent to equation (3). Then, the relationship between x _{n, k} , d _{n, k} and C _n can be derived as follows based on the equations (4), (11) and (12).

Expression (13) is a conditional joint probability density function of the spectrum feature quantity of the observation signal and the occupying sound source number at each frequency with the spectrum shape feature quantity of all the sound sources being given. Represents a function defined by the product of the probability functions of.
1. Probability function (first element on the right side) when it is specified that the spectrum feature of the sound source has the same value as the spectrum feature of the observed signal in the spectrum observation model of the sound source that matches the occupied sound source number
2. In a spectrum observation model of a sound source other than the sound source that matches the occupied sound source number, a probability function (second right side of the right side) is defined that the spectrum feature amount of the sound source is less than or equal to the value of the spectrum feature amount of the observation signal. Each integral term in the element)

Finally, using equation (13), the MAP function p (x _n , C _n ) in equation (8) is defined as follows.

Here, d _{n, k} is treated as a hidden variable. The first term on the right side of Equation (14) corresponds to the spectrum observation model and the LogMax model, and the second term on the right side of Equation (14) corresponds to the spectrum shape model of each sound source. In the present technology, the spectral shape feature quantity that maximizes the expression (14) is obtained for all sound sources, thereby realizing estimation in consideration of all of the spectral shape model, the spectrum observation model, and the LogMax model.

More specifically, the equation (14) is a simultaneous probability density function of the spectral feature quantity x _n of the observed signal and the spectral shape feature quantity C _n of all sound sources, and consists of the product of two elements as shown on the right side. The first element on the right side of the equation (14) is a conditional probability density function (that is, p (x _n | C _n ) of the spectral features of the observed signal given the spectral shape features of all sound sources. ). Hereinafter, this is referred to as an observation signal spectrum observation model. The second element on the right side of Equation (14) is the product of spectral shape models (ie, Π _l p (c _n ^(l) )), which is a prior probability density function of spectral shape features determined for each sound source. is there.
The observation signal spectrum observation model has the following characteristics as shown on the right side of Expression (14).
1. The observed signal spectrum observation model can be decomposed into products of conditional probability density functions corresponding to the respective frequencies (that is, p (x _n | C _n ) = Π _k p (x _{n, k} | C _n )).
2. The conditional probability density function corresponding to each frequency has an occupying sound source number indicating which sound source is the most occupying sound source at that frequency as a latent variable. (Ie, p (x _{n, k} | C _n ) = Σ _d p (x _{n, k} , d _{n, k} | C _n ))
3. The conditional probability density function corresponding to each frequency is determined as shown in Expression (13) based on the spectrum observation model and the LogMax model determined for each sound source.

  The present technology estimates the spectral shape feature amount of each sound source that maximizes Expression (14), and estimates the sound source occupancy of each sound source, which is a posterior probability function in which each sound source matches the occupying sound source number. Further, the spectrum of the target signal is estimated based on the spectral shape feature amount and the sound source occupancy estimated for the target signal and the spectrum observation model of the target signal.

[Optimization algorithm]
As an algorithm for maximizing Expression (14), a general nonlinear optimization algorithm such as a conjugate gradient method or a quasi-Newton method can be applied. Since these methods can be derived by a general method based on each nonlinear optimization algorithm, description thereof is omitted here.

On the other hand, since Equation (14) is a function including a hidden variable, it can be efficiently maximized by using an expectation maximization (EM) algorithm. In the following, this optimization algorithm will be described in detail. The auxiliary function used in the EM algorithm is defined as follows.

Now, if the estimated value of the sound source occupancy of the sound source m is expressed as M ^ _{n, k} ^(m) = E {p (d _{n, k} = m | x _{n, k} , C _n ) | C ^ _n } (17) can be further expanded as follows.

According to the EM algorithm, Eq. (18) is calculated in E-step, C _n that maximizes Eq. (19) is obtained in M-step, and the above optimization is performed by repeating E-step and M-step. A spectral shape feature quantity C _n that maximizes the function can be obtained.

  Note that in E-step, equation (18) can be calculated independently at each frequency. For this reason, it is not necessary to consider a combination of values between frequencies in updating the sound source occupancy. Further, in M-step, equation (19) can be maximized by decomposing into an optimization function for each sound source, as in equation (20). Therefore, according to the equation (20), the spectral shape feature amount can be updated independently for each sound source. For this reason, it is not necessary to consider the combination of values between sound sources. As a result, the sound source occupancy and the spectral shape feature amount of the sound source m can be efficiently updated based on the repeated estimation of the EM algorithm.

[Estimation of spectral features of target signal]
The target sound spectrum estimation unit 106 includes the spectral feature amount x _{n, k} of the observation signal, the estimated value M ^ _{n, k} ⁽¹⁾ of the sound source occupancy of the target signal, and the estimated value c ^ _n ^{( 1)} and the spectrum observation model p (s _n ⁽¹⁾ | c _n ⁽¹⁾ ) are used as inputs, and an estimated value s ^ _{n, k} ⁽¹⁾ of the target signal spectrum is obtained by least square error estimation. The estimation method is as follows.

[Embodiment 1]
Based on the above description, in the present technology, the target sound spectrum can be estimated by the following procedure.
Step 1. For each short-time frame n, the feature extraction unit 104 extracts the spectral feature amount x _n of the observation signal.
Step 2. The spectrum shape estimation unit 103-m corresponding to each sound source m initializes an estimated value c ^ _n ⁽¹⁾ of the spectrum shape feature quantity. For example, it is assumed that c ^ _n ⁽¹⁾ = H (x _n ) using the spectral feature amount x _n of the observed signal and the feature amount conversion function H (•).
Step 3. Repeat (a) and (b) until convergence.
(a) The sound source occupancy update unit 105 updates the estimated value M ^ _{n, k} ^(m) of the sound source occupancy of each sound source m by calculating Equation (18) independently for each frequency ( E-step).
(b) For each sound source m, the spectrum shape estimation unit 103-m updates the estimated value c ^ _n ^(m) of the spectrum shape feature amount by obtaining c _n ^(m) that maximizes Equation (20). (M-step). At this time, since the equation (20) is generally a nonlinear function, the maximization is realized by a general nonlinear optimization method such as a conjugate gradient method, a quasi-Newton method, or a Newton method.
Step 4. The target sound spectrum estimation unit 106 obtains an estimated value s ^ _{n, k} ⁽¹⁾ of the spectral feature quantity of the target signal according to the equation (21).

As an example, the above 3. In (b), the update formula for updating the spectrum shape feature quantity according to the Newton method is as follows.

[Embodiment 2]
A more specific embodiment will be described from the first embodiment. First, it is assumed that the output of the mel filter bank is used as the spectral feature amount of the observation signal, MFCC is used as the spectral shape feature amount, and the mixed gaussian distribution of MFCC is used as the spectral shape model.

It is assumed that the spectral shape feature value c _n ^(m) of each sound source m is modeled as follows using a mixed Gaussian distribution. Here, p (c _n ^(m) | i) and α _i represent the Gaussian distribution and the mixture ratio for the mixture number i, respectively. It is assumed that the average μ _i ^(m) and the covariance matrix Σ _i ^(m) of each Gaussian distribution are obtained in advance using a database of acoustic signals. In this model, the mixture number i is treated as a hidden variable.

In the second embodiment, the biggest difference from the first embodiment is that the mixture number i of the mixed Gaussian distribution in the spectrum shape model of each sound source m is newly included as a hidden variable of the EM algorithm. Accordingly, the auxiliary function used in the EM algorithm is modified as follows.

Further, when _expressed as Z ^ _{n, i} ^(m) = E {p (i | c _n ^(m) ) | c ^ _n ^(m) }, equation (20) is modified as follows.

As a result, the target sound spectrum can be estimated by the following procedure.
Step 1. For each short-time frame n, the feature extraction unit 104 extracts the mel filter bank output x _n related to the observation signal.
Step 2. The spectrum shape estimation unit 103-m corresponding to each sound source m initializes the estimated value c ^ _n ^(m) of the MFCC. For example, H (x _n ) is a discrete cosine transform, and c ^ _n ^(m) = H (x _n ).
Step 3. Repeat (a), (b) and (c) until convergence.
(a) The sound source occupancy update unit 105 updates the estimated value M ^ _{n, k} ^(m) of the sound source occupancy of each sound source m by calculating Equation (18) independently for each frequency ( E-step1).
(b) For each sound source m, the spectrum shape estimation unit 103-m updates Z ^ _{n, i} ^(m) based on the equation (28) (E-step 2).
(c) For each sound source m, the spectrum shape estimation unit 103-m calculates the MFCC c _n ^(m) that maximizes Equation (27), thereby updating the estimated value c ^ _n ^(m) of the MFCC. At this time, since the equation (27) is generally a nonlinear function, its maximization is realized by a general nonlinear optimization method such as a conjugate gradient method, a quasi-Newton method, and a Newton method (M-step). ).
Step 4. The target sound spectrum estimator 106 obtains an estimated value s ^ _{n, k} ^(m) of the filter bank output (= spectrum feature quantity) of the target signal using equation (21).

Next, in the second embodiment, a case where a more specific model is used for the spectrum observation model p (s _{n, k} ^(m) | c _n ^(m) ) for each sound source m will be described.

In the second embodiment, the pseudo inverse matrix G (c) is modeled with a linear regression model defined by the following. However, A represents a matrix (N (k) × N (m)), and b represents a vector (N (k) × 1).
G (c) = Ac + b (29)

The values of A and b are learned in advance from a database of acoustic signals or learned using observation signals. That is, when a combination of a plurality of spectral feature amounts x _n and corresponding spectral shape feature amounts c _n = H (x _n ) is given from the learning database (or observation signal), A and b Shall be determined as follows:

The value of the variance ξ _k of the inverse transformation error e is the mean square regression error E {| x _n − (Ac _n + b) | ² } obtained by using A and b obtained by the equation (30). Suppose that it is determined as a value corresponding to each frequency k. Based on the above definition, the spectrum observation model p (s _{n, k} ^(m) | c _n ^(m) ) can be determined by equation (10). An individual spectrum observation model may be prepared for each sound source based on data that can be used for estimation of A and b, or a common spectrum observation model may be prepared for all sound sources.

  The algorithm in this case only changes the definition of the pseudo inverse transform G (c) as compared with the algorithm of the second embodiment.

  As another example of the definition of G (c), it is also possible to define A as a Moore-Penrose pseudo inverse transformation matrix for a discrete cosine transformation matrix, b = 0. This means that among the inverse transforms of the discrete cosine transform, the one that minimizes the norm of the transformed feature value is selected. As described above, in the present technology, inverse transformation based on various criteria can be adopted as the pseudo inverse transformation G (c).

[Modification of Embodiment 2]
In the second embodiment, a case where a logarithmic power spectrum is used as an observation signal or a spectrum feature amount of each sound source will be described. This can be realized by simply replacing the process related to the mel filter bank output with the process related to the logarithmic power spectrum in the above algorithm. Based on the above algorithm, the changes are summarized as follows.
= 1 =
Step 1. The feature extraction unit 104 extracts a logarithmic power spectrum as the spectral feature amount x _n of the observation signal instead of the mel filter bank output.
= 2 =
Step 2. In the initialization, the function H (x _n ) = log (mfb (exp (x _n ))) for converting the logarithmic power spectrum to MFCC, which is given as an example, is used as the feature amount conversion function.
= 3 =
Step 3. (a), 3. It is assumed that the spectrum observation model for each sound source m used in (c) is determined based on an inverse transform G (c) for obtaining a logarithmic power spectrum from the MFCC.
= 4 =
Step 4. In, the logarithmic power spectrum is estimated as the spectrum estimation value of the target signal. For this reason, the logarithmic power spectrum of the observation signal is used as the spectral feature quantity x _n of the observation signal, and step 3. (a), Step 3. As in (c), the spectrum observation model is determined based on the inverse transform G (c) for obtaining the logarithmic power spectrum from the MFCC.

In the above algorithm, what is finally obtained is an estimated value s ^ _{n, k} ^(m) of the logarithmic power spectrum of the target signal. Therefore, the estimated value of the power spectrum of the target signal can also be obtained from here as exp (s ^ _{n, k} ^(m) ). Furthermore, using the phase information of the observation signal, an estimated value of the waveform of the target signal can be obtained by discrete inverse Fourier transform and overlap addition synthesis technology.

[Embodiment 3]
In the second embodiment, instead of using a mixed Gaussian distribution related to MFCC as a spectral shape model, a case of using a hidden Markov model related to MFCC is configured. In this embodiment, for the sake of simplicity, description will be made on the premise of batch processing in which all estimation processing is performed after all observation signals are given. However, it is possible to operate the hidden Markov model by sequential processing using known techniques, and it is also possible to operate this embodiment sequentially using these techniques.

First, _let {c _n ^(m) } denote a set of c _n ^(m) for all short-time frames n. The spectrum shape model p ({c _n ^(m) }) of each sound source m can be modeled as follows using a hidden Markov model. Where i ₀ is the initial state, i _n for n> 0 is the state in the short time frame n, π _i0 is the probability that the initial state is i _0, and π _{i, j} is the transition probability from state i to state j ^{_{, p (c n (m)}} | i) is the output probability density function of the spectral shape feature in the state i, sigma _I represents the sum concerning all combinations of state sequences.

In the hidden Markov model, given a sequence of observed features, the posterior probability function Z _{n, i} = p (i _n = i | {c _n ^(m) }) taking each state i in each short frame is It is known that it can be obtained efficiently using a forward-backward algorithm, Viterbi algorithm, or the like.

When the hidden Markov model is used for the spectral shape model, the auxiliary function used in the EM algorithm is modified as follows.

Furthermore, when _expressed as Z ^ _{n, i} ^(m) = E {p (i _n | {c _n ^(m) }) | {c ^ _n ^(m) }}, the equations (19) and (20) are expressed as follows. It is corrected as follows.

Therefore, the main difference from the auxiliary function of the second embodiment is only the calculation method of Z ^ _{n, i} ^(m) in Expression (35). Z ^ _{n, i} ^(m) is a hidden Markov model whose output value is the estimated value {c ^ _n ^(m) } of the spectral shape feature that was updated in M-step. Since it is the same as the posterior probability p (i _n | {c ^ _n ^(m) }), it can be efficiently obtained using the forward-backward algorithm, the Viterbi algorithm, or the like as described above.

Finally, the algorithm difference from the second embodiment is as follows. The process of (b) is only corrected as follows.
Step 3. (b) For each sound source m, the spectrum shape estimation unit 103-m uses a forward-backward algorithm or the like, and Z ^ _{n, i} ^(m) = p (i _n | {c ^ _n ^(m) }) Ask for.

[Embodiment 4]
In the embodiments so far, it is assumed that the spectral shape model is given in advance for all sound sources, but such a situation cannot be expected in an actual environment. For example, the spectral shape model of the target signal can be learned in advance, but the spectral shape model of the background noise cannot be learned in advance. In this case, it is necessary to learn a spectrum shape model of a sound source that has not been pre-learned by only information obtained from the observation signal. Embodiment 4 will be described as a method for dealing with such a situation.

In the fourth embodiment, only the spectral shape feature value c _n ^(m) is estimated as a parameter to be obtained by the EM algorithm in the second and third embodiments, whereas the parameter of the mixed Gaussian distribution that is a spectral shape model is used. (Mixed ratio, average and covariance of each Gaussian distribution) and hidden Markov model parameters (initial state probability, state transition probability, average and covariance of output probability distribution) are included in the parameters to be estimated. First, the problem of obtaining the spectral shape feature value is defined as a problem that maximizes the following optimization function. Here, θ is a set including all parameters to be obtained, and it is assumed that unknown parameters of the spectrum shape model are included in addition to the spectrum shape feature amount C _n .

Here, an algorithm when the parameters of the spectrum shape model are not given in advance for one or more sound sources will be described. For the sake of simplicity, only differences from the algorithms of the second and third embodiments will be described. As in the case of the modification of the third embodiment with respect to the second embodiment, the modification of the algorithm is performed in step 3. Only for (b). Specifically, in the fourth embodiment, for the sound source m for which the spectral shape model is not given in advance, step 3. The process (b) is composed of the following two processes.
Step 3. (b) -1: First, parameters of the spectrum shape model are estimated using c ^ _n ^(m) updated in the initialization process or M-step. That is, the parameters of the spectrum shape model are estimated so that the likelihood that c ^ _n ^(m) is generated as the spectrum shape feature amount is maximized. For the parameter estimation, an algorithm suitable for the probability distribution adopted by the spectral shape model may be used. For example, in the case of a mixed Gaussian distribution or a hidden Markov model, it is known that parameter estimation can be performed efficiently using an EM algorithm.
Step 3. (b) -2: After estimating the parameters of the spectral shape model, Z ^ _{n, k} ^(m) is _obtained by the same procedure as in the second and third embodiments.

  Even if the spectral shape model is not given in advance by the above processing, step 3. The process (b) can be performed. Further, in the target signal spectrum estimation processing other than the above, since the spectrum shape model obtained above can be used, the algorithm is not particularly changed.

The parameter estimation of the spectral shape model described above is performed so that the likelihood that the spectral shape model outputs the estimated value given the estimated value c ^ _n ^(m) of the spectral shape feature amount. The Therefore, the spectrum shape model that can be used in the configuration of the fourth embodiment is not limited to the above-described one, and includes any spectrum shape model that allows parameter estimation based on a given output sequence.

[Embodiment 5]
As in the conventional example shown in Non-Patent Document 1-3, the present technology takes a configuration in which the sound source occupancy is repeatedly estimated while being updated with the E-step of the EM algorithm. Therefore, if the observation signal is recorded simultaneously from multiple microphones, the feature value based on the position information of each sound source is extracted from the observation signal based on the same method as the conventional example, and the sound source position is It is possible to adopt a configuration in which the sound source occupancy is updated in consideration of this information. In the present embodiment, this configuration will be described.

Hereinafter, this embodiment will be described with reference to FIG. In the figure, in order to distinguish a plurality of microphone signals, a microphone number is written on the right shoulder as x ⁽¹⁾ (t). For the sake of simplicity, the case where the number of microphones is two is illustrated, but any number of microphones may be used as long as the number of microphones is two or more as in the conventional example. The feature extraction unit 104 extracts the spectral feature amount x _n of the observation signal from any one of the observation signals, as in the previous embodiments. However, it is also possible to employ a configuration in which a plurality of microphones are used to extract a spectral feature amount obtained as a result of applying some processing (for example, enhancing a target sound using another speech enhancement technique). When the spectrum shape model, the spectrum observation model, the spectrum shape estimation unit, and the target sound spectrum estimation unit for each sound source are the same as those described in the embodiments so far, or those configured in a similar manner are used. To do. In the present embodiment, descriptions of these functional units are omitted unless particularly necessary.

  On the other hand, in the present embodiment, as a new functional unit, the spectral shape feature quantity estimation device 100p (that is, the target signal spectral feature quantity estimation device 100) includes a sound source position feature extraction unit 111 and a sound source position associated with each sound source m. A state estimation unit 112-m is provided. In the figure, the case where the number of sound source position state estimation units 112-m is two is illustrated, but actually, it is assumed that the sound source position state estimation units 112-m are prepared according to the number of sound sources. . As in the previous embodiments, m = 1 is the sound source position state estimation unit 112-m corresponding to the target sound.

The sound source position feature extraction unit 111 generates sound source position feature quantities a _n = [a _{n, 1} , a _{n, 2} ,..., A _{n, N (k} from observation signals x ^(m) (t) from a plurality of microphones. ₎ ] Extract ^T. Here, a _{n, k} is the sound source position feature quantity at the time frequency point (n, k). Similar to Non-Patent Documents 1-3 and the like, as the sound source position feature amount, one having a tendency to take different values depending on the sound source position may be adopted. For example, a phase difference between microphones, a time difference between microphones, a microphone There are various options such as the difference in strength. In this specification, the case where the following normalized complex spectrum is employ | adopted as a sound source position feature-value as an example is demonstrated. Where X _{n, k} = [X _{n, k} ⁽¹⁾ , X _{n, k} ⁽²⁾ , ..., X _{n, k} ^{(N (m))} ] ^T and X _{n, k} ^(m) is This is the kth frequency element of the short-time Fourier transform of x ^(m) (t) corresponding to the short-time frame n. | · | Represents the norm of the vector.
a _{n, k} = X _{n, k} / | X _{n, k} | (37)

Next, each sound source position state estimation unit 112-m obtains the following values based on the sound source position feature quantity an _{, k} and the estimated value φ ^ ^{(m) of the} sound source position parameter. p (a _{n, k} ; φ ^(m) ) represents the probability density function related to the sound source position feature of the sound source m at the frequency k. In the following, w ^ _{n, k} ^(m) is the sound source position state of the sound source m. This is called an estimated value.
w ^ _{n, k} ^(m) = p (a _{n, k} ; φ ^(m) ) (38)

Various probability density functions related to the sound source position feature quantity are known depending on the sound source position feature quantity. For example, the probability density function in the case where the normalized complex spectrum described above is used as the sound source position feature amount is described in Reference Document 1 and the like.
(Reference 1) Hiroshi Sawada, Shoko Araki, and Shoji Makino, “Underdetermined convolutive blind source separation via frequency bin-wise clustering and permutation alignment,” IEEE Trans. Audio, Speech, and Language Processing, vol. 19, No. 3 , pp.516-527, 2011.

The estimated value φ ^ ^(m) of the sound source position parameter is learned in advance using the acoustic signals collected by multiple microphones under the same recording conditions as the observation signal as learning data, or directly from the observation signal. be able to. Various learning methods are known depending on the sound source position feature quantity. For example, the learning method in the case where the normalized complex spectrum shown above is used as the sound source position feature amount is detailed in Reference Document 1. In addition, in the target signal spectrum estimation of the present technology, each sound source position state estimation unit 112-m performs the sound source in each iteration in parallel with the update of the estimated value of the sound source occupancy and the estimated value of the spectral shape feature amount. It is also possible to receive the estimated value of the sound source occupancy degree from the occupancy degree updating unit 105 and update the estimated value of the sound source position parameter. Non-Patent Document 2 describes a method for repeatedly updating an estimated value of a sound source position parameter in target signal spectrum estimation.

Further, the sound source occupancy update unit 105 receives the spectral feature amount x _n of the observation signal, and estimates the spectral shape feature amount c ^ _n ^(m) and the spectral observation model p (s _n ^{(m )} | c _n ^(m) ) In addition to receiving the estimated position w ^ _{n, k} ^(m) of each sound source m, the estimated sound source occupancy M ^ _{n, k} Update ^(m) . Similar to Non-Patent Document 2, this update can be realized as follows.

As a modification when the logarithmic power spectrum is used as the spectrum feature amount in the second embodiment, the algorithm of the present embodiment is summarized as follows.
Step 1. Feature extraction is performed by the processes (a) and (b).
(a) For each short-time frame n, the feature extraction unit 104 extracts a logarithmic power spectrum x _n related to the observation signal.
(b) The sound source position feature extraction unit 111 extracts a sound source position feature amount related to the observation signal by Expression (37).
Step 2. (a) Initialization is performed by the processing of (b).
(a) The spectrum shape estimation unit 103-m corresponding to each sound source m initializes an estimated value c ^ _n ^(m) of MFCC. For example, H (x _n ) = log (mfb (exp (x _n ))) and c ^ _n ^(m) = H (x _n ).
(b) For each sound source m, the sound source position state estimation unit 112-m obtains an estimated value w ^ _{n, k} ^(m) of the sound source position state using Equation (38).
Step 3. Repeat (a)-(c) until convergence.
(a) The sound source occupancy update unit 105 updates the estimated value M ^ _{n, k} ^(m) of the sound source occupancy of each sound source m by calculating Equation (39) independently for each frequency ( E-step1).
(b) For each sound source m, the spectrum shape estimation unit 103-m updates Z ^ _{n, i} ^(m) based on the equation (28) (E-step 2).
(c) For each sound source m, the spectrum shape estimation unit 103-m calculates the MFCC c _n ^(m) that maximizes Equation (27), thereby updating the estimated value c ^ _n ^(m) of the MFCC. At this time, since the equation (27) is generally a nonlinear function, its maximization is realized by a general nonlinear optimization method such as a conjugate gradient method, a quasi-Newton method, and a Newton method (M-step). ).
Step 4. The target sound spectrum estimation unit 106 obtains an estimated value s ^ _{n, k} ^(m) of the logarithmic power spectrum (= spectrum feature amount) by the equation (21).

[Confirmation experiment]
A confirmation experiment was conducted to evaluate the target signal spectrum estimation technique. The experimental conditions will be described. In a room with reverberation, we used two microphones and used the sound of the speaker in front of the microphone as well as various surrounding background sounds. Using this observation signal, a comparative experiment of the target signal spectrum estimation method shown in Non-Patent Document 3 (conventional example) and Embodiment 5 (present technology) was performed. Both the conventional example and this technology use the logarithmic power spectrum as the spectral feature of the observed signal. In this technology, MFCC is used as the spectral shape feature, and a Gaussian mixture model is used as the spectral shape model. In addition, in both the conventional example and the present technology, the same sound source position feature amount model as in Non-Patent Document 3 is used, and parameters thereof are prepared by prior learning.

  The result of applying automatic speech recognition to a signal obtained by estimating the spectrum of an observation signal and a target signal is shown. The correct word rate when the observed signal was recognized as speech was 69.4%, whereas the correct word rate when applying speech recognition to the target signal spectrum estimated by the conventional example and this technology is 83.7% and 87.2%. Even in the conventional example, a significant improvement in the speech recognition rate has been obtained, but a further significant improvement has been obtained by this technology.

  From the above results, it was confirmed that the present technology can significantly improve the spectrum estimation accuracy of the target signal by using a mixed Gaussian distribution related to MFCC as a spectrum shape model compared to the conventional example.

<Hardware configuration example of spectrum shape feature quantity estimation device / target signal spectrum feature quantity estimation device>
The spectral shape feature quantity estimation apparatus / target signal spectral feature quantity estimation apparatus (hereinafter simply referred to as estimation apparatus) according to the above-described embodiment includes an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [A cache memory may be provided. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. If necessary, the estimation apparatus may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the estimation device stores a program for estimating a spectrum shape feature amount and the like and data necessary for processing of this program [not limited to the external storage device, for example, a program is read-only stored. You may memorize | store in the ROM which is an apparatus. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  In the storage unit of the estimation device, a program for extracting a spectral feature amount from an observation signal, a program for estimating a most likely spectral shape feature amount based on an optimization function (specifically, a spectral shape feature amount) , A program for updating the sound source occupancy), a program for estimating the spectral feature quantity of the target signal, a program for estimating the sound source position feature quantity if necessary, and a sound source A program for estimating the position state is stored.

  In the estimation apparatus, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and interpreted and executed by the CPU. As a result, spectrum estimation is performed by the CPU realizing predetermined functions (feature extraction unit, spectrum shape estimation unit, sound source occupancy update unit, target sound spectrum estimation unit, sound source position feature extraction unit, sound source position state estimation unit). Realized. The target sound spectrum estimation unit is not an essential component of the spectrum shape feature quantity estimation device.

<Supplementary note>
The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  When the processing functions in the hardware entity (estimation device) described in the above embodiment are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (11)

A spectral shape feature amount estimation device for estimating a spectral shape feature amount representing a shape of a spectral feature amount of each acoustic signal from an observation signal obtained by mixing and collecting acoustic signals from a plurality of sound sources,
Stores the prior probability density function (spectral shape model) of spectral shape features corresponding to each of the above sound sources and the conditional probability density function (spectral observation model) of spectral features given the spectral shape features A storage unit to
From the spectral feature quantity of the observed signal, the prior probability density function (spectral shape model) of the spectral shape feature quantity, and the conditional probability density function (spectral observation model) of the spectral feature quantity given the spectral shape feature quantity An estimation means for estimating a spectral shape feature amount for each sound source that maximizes an optimization function having a dedicated sound source number representing a sound source of an acoustic signal having the maximum energy at each time frequency point as a latent variable. Including
The above spectral shape model is a probability density function having a correlation between frequencies,
The optimization function includes a conditional probability density function of the spectral features of the observed signal given the spectral shape features of all the sound sources, and a prior probability density function of the spectral shape features determined for each of the sound sources. A spectral shape feature quantity estimation device expressed by the product of The spectral shape feature amount estimation apparatus according to claim 1,
The estimation means is
Conditional probability density function (spectrum observation model) of the spectral feature amount for each of the above sound sources, the spectral feature amount of the observation signal, and the spectral feature amount for each of the above sound sources. And estimating the posterior probability (sound source occupancy) that each sound source is a sound source represented by the exclusive sound source number,
Estimated posterior probability (sound source occupancy), spectral feature quantity of observed signal, prior probability density function of spectral shape feature quantity (spectral shape model), and spectral feature quantity condition given spectral shape feature quantity A spectral shape feature quantity estimation device for estimating a spectral shape feature quantity for each sound source from a probability density function (spectral observation model) with a mark. The spectral shape feature amount estimation apparatus according to claim 2,
The spectral shape feature amount is a mel frequency cepstrum coefficient,
The spectral shape feature amount estimation apparatus, wherein the spectral shape model is a mixed Gaussian distribution of mel frequency cepstrum coefficients. The spectral shape feature amount estimation apparatus according to claim 2 or 3,
A sound source position feature extraction unit that extracts a sound source position feature amount that is a feature amount related to the position of each sound source from the observation signals obtained by each of a plurality of microphones;
A sound source position state estimation unit for obtaining an estimated value of a sound source position state represented by a probability density function related to the sound source position feature amount for each sound source;
Said estimating means, the spectral shape feature quantity estimation apparatus characterized by updating the source occupancy also using the estimate of the sound source position state of said each sound source. The spectrum shape feature quantity corresponding to the sound source of the target signal out of the spectrum shape feature quantity and the sound source occupancy for each sound source estimated by the spectrum shape feature quantity estimation device according to any one of claims 2 to 4; A target signal including a target sound spectrum estimator that estimates a spectral feature of the target signal from the sound source occupancy of the target signal, a spectrum observation model corresponding to the target signal, and a spectral feature of the observed signal Spectral feature quantity estimation device. A spectral shape feature amount estimation method for estimating a spectral shape feature amount representing a shape of a spectral feature amount of each acoustic signal from an observation signal obtained by mixing acoustic signals from a plurality of sound sources,
The storage unit has a prior probability density function (spectral shape model) of spectral shape features corresponding to each of the above sound sources, and a conditional probability density function (spectrum observation) of the spectral features given the spectral shape features. Model)
From the spectral feature quantity of the observed signal, the prior probability density function (spectral shape model) of the spectral shape feature quantity, and the conditional probability density function (spectral observation model) of the spectral feature quantity given the spectral shape feature quantity Including an estimation step for estimating a spectrum shape feature amount for each sound source that maximizes an optimization function having an exclusive sound source number representing a sound source of an acoustic signal having the maximum energy at each time frequency point as a latent variable. ,
The above spectral shape model is a probability density function having a correlation between frequencies,
The optimization function includes a conditional probability density function of the spectral features of the observed signal given the spectral shape features of all the sound sources, and a prior probability density function of the spectral shape features determined for each of the sound sources. A spectral shape feature amount estimation method expressed by the product of The spectral shape feature amount estimation method according to claim 6,
The estimation step is:
Conditional probability density function (spectrum observation model) of the spectral feature amount for each of the above sound sources, the spectral feature amount of the observation signal, and the spectral feature amount for each of the above sound sources. And estimating the posterior probability (sound source occupancy) that each sound source is a sound source represented by the exclusive sound source number,
Estimated posterior probability (sound source occupancy), spectral feature quantity of observed signal, prior probability density function of spectral shape feature quantity (spectral shape model), and spectral feature quantity condition given spectral shape feature quantity A spectral shape feature amount estimation method, wherein a spectral shape feature amount for each sound source is estimated from a probability density function (spectral observation model) with a threshold. The spectral shape feature amount estimation method according to claim 7,
The spectral shape feature amount is a mel frequency cepstrum coefficient,
The spectral shape feature quantity estimation method, wherein the spectral shape model is a mixed Gaussian distribution of mel frequency cepstrum coefficients. The spectral shape feature amount estimation method according to claim 7 or 8,
A sound source position feature extraction step of extracting a sound source position feature amount that is a feature amount related to the position of each sound source from the observation signals obtained by each of a plurality of microphones;
A sound source position state estimation step for obtaining an estimated value of a sound source position state represented by a probability density function related to the sound source position feature amount for each sound source,
The estimation step includes the step of updating the sound source occupancy degree using the estimated value of the sound source position state of each sound source, and a spectral shape feature quantity estimation method. The spectrum shape feature quantity corresponding to the sound source of the target signal among the spectrum shape feature quantity and the sound source occupancy for each sound source estimated by the spectrum shape feature quantity estimation method according to claim 7. A target signal including a target sound spectrum estimation step for estimating a spectral feature of the target signal from the sound source occupancy of the target signal, the spectrum observation model corresponding to the target signal, and the spectral feature of the observed signal Spectral feature estimation method.       A program for causing a computer to function as the spectrum shape feature quantity estimation device according to any one of claims 1 to 4 or the target signal spectrum feature quantity estimation device according to claim 5.

Images