JP6059112B2 - Sound source separation device, method and program thereof - Google Patents

Description

  The present invention relates to a sound source separation device that extracts each target signal with high accuracy when an input signal includes a plurality of target signals, and a method and program thereof.

  When an acoustic signal is collected in an environment where a plurality of target sound sources exist, a mixed signal in which the target signals overlap each other is often observed. At this time, when the target sound source of interest is an audio signal, the clarity of the target sound is greatly reduced due to the influence of other sound source signals superimposed on the target signal. As a result, it becomes difficult to extract the nature of the original target speech signal (hereinafter referred to as the target signal), and the recognition rate of the automatic speech recognition (hereinafter referred to as speech recognition) system is significantly reduced. Therefore, in order to prevent the recognition rate from being lowered, it is necessary to devise a method (method) for recovering the clarity of the target signal by separating a plurality of target signals.

  The elemental technology for separating the plurality of target signals can be used for various acoustic signal processing systems. For example, a hearing aid that extracts the target signal from the sound collected in the real environment to improve ease of hearing, a TV conference system that improves the intelligibility of the voice by extracting the target signal, and audio used in the real environment It can be used for a recognition system, a machine-human interaction device in a machine control interface, a music information processing system for searching and recording music, and the like.

  FIG. 7 shows a functional configuration of a conventional sound source separation device 900 disclosed in Non-Patent Document 1, for example, and its operation will be briefly described. The sound source separation device 900 includes a sound source existence posterior probability estimation unit 90 and a filtering unit 91 for all microphones.

  The sound source signal posterior probability estimation unit 90 common to all microphones receives a plurality of channel observation signals obtained by collecting sound source signals emitted from a plurality of sound sources with a plurality of microphones, and characterizes each time frequency bin of each observation signal And the existence probability for each sound source is calculated by classifying the feature vectors. The filtering unit 91 recovers the sound source signal by multiplying the observation signals of a plurality of channels collected by a plurality of microphones by the existence probability.

H. Sawada, S. Araki, and S. Makino, “Underdetermined convolutive blind source separation via frequency bin-wise clustering and permutation alignement,” IEEE Trans. Audio, Speech and Lang. Process., Vol. 19, pp.516- 527, March 2011.

  However, if a plurality of microphones are arranged in a spatially dispersed manner, the sound pressure of a certain sound source observed by each microphone does not become comparable. In extreme cases, a situation can occur in which a certain sound source is substantially unobservable with a certain microphone. In such a situation, it is reasonable to assume different sound source existence probabilities (activity patterns) for each microphone. However, in the conventional method, since the sound source existence probability cannot be calculated for each microphone, there is a problem that efficient sound source separation cannot be performed in a distributed microphone array environment.

  The present invention has been made in view of such a problem, and an object thereof is to provide a sound source separation apparatus, a method thereof, and a program capable of efficiently performing sound source separation even in a distributed microphone array environment.

  The sound source separation device according to the present invention includes a microphone-specific sound source presence posterior probability estimation unit, a model parameter estimation unit, and an output sound estimation unit. The microphone-specific sound source existence posterior probability estimation unit includes a plurality of channel observation signals obtained by collecting sound source signals emitted from a plurality of sound sources by a plurality of microphones, and each of the plurality of sound sources observed by each of the plurality of microphones. The sound source existence posterior probability for each sound source is estimated for each microphone, using the model of the observed signal assuming that the sound pressures of the signals are different. The model parameter estimation unit estimates the model parameters of the observation signal by using the observation signals of a plurality of channels and the sound source existence posterior probability as inputs. The output sound estimation unit estimates and outputs an incoming signal from each sound source for each microphone by using the observation signals of a plurality of channels, the sound source posterior probability, and the model parameters as inputs.

  According to the sound source separation device of the present invention, the arrival signal (sound source image) from the sound source is estimated for each sound source using the sound source existence posterior probability estimated for each sound source for each of a plurality of microphones. Sound source separation can be performed efficiently. Specific effects confirmed in the evaluation experiment will be described later.

The figure which shows the function structural example of the sound source separation apparatus 100 of this invention. The figure which shows the operation | movement flow of the sound source separation apparatus. The figure which shows the function structural example of sound source separation apparatus 100 'using EM algorithm and Newton-Raphson method of this invention. The figure which shows the operation | movement flow of model parameter optimization. The figure which shows the acoustic environment used for evaluation experiment. Figure showing the evaluation experiment results The figure which shows the function structural example of the conventional audio | voice separation apparatus 900.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. Before the description of the embodiment, the observation signal is modeled.

[Modeling of observed signals]
When sound generated from a plurality of point sound sources (1, 2,... N _i ) is observed by the m-th microphone of the plurality of microphones (1, 2,... N _m ), the signal x coming from the i-th sound source _{t, f} ^{(i, m)} is expressed as follows in the time-frequency domain. t (t = 1,... N _t ) and f (f = 1,..., N _f ) are time and frequency indexes.

Here, S _{t, f} ⁽ⁱ⁾ and _{st, f} ⁽ⁱ⁾ correspond to a signal in the short-time Fourier transform domain and a signal in the logarithmic power domain of the clean speech signal from the i-th sound source, respectively. This is a microphone position independent parameter. Similarly, H _f ^{(i, m)} and β _f ^{(i, m)} correspond to transfer functions in the short-time Fourier transform region and the logarithmic power spectrum region.

In the following description, the variable β _f ^{(i, m)} is referred to as a microphone position dependent / sound source time invariant gain. A signal x _{t, f} ^{(i, m)} coming from the i-th sound source is referred to as a sound source image. ^{_{e t, f (i, m}} ) is the error ^{_{term, x t, f (i,}} m) and ^{_{log | S t, f (i}} ) H f (i, m) | is the difference between the ^2, for example, transfer Represents function fluctuations. This error term _{et, f} ^{(i, m)} is assumed to be a white signal with an average of 0 and a variance σt _{, f} ^{(i, m)} .

According to the above definition, the relationship between the clean sound signal s _{t, f} ⁽ⁱ⁾ from the i-th sound source and the sound source image x _{t, f} ^{(i, m)} is as follows as a probability density function of Gaussian distribution. Can be modeled.

Here, θ ⁽ⁱ⁾ represents a set of model parameters. N means a normal distribution.

Next, the observation signal o _{t, f} ^(m) collected by the m-th microphone in an environment where a plurality of point sound sources exists is modeled using LogMax approximation. If the approximation is used, the observed signal ot _{, f} ^(m) becomes the same value as the dominant sound source signal having the maximum sound pressure among all point sound sources as shown in the following equation.

  A sound source that is not dominant in this modeling can take any value as long as it is a value less than or equal to the logarithmic power spectrum of the observation signal. The above LogMax approximate model is stochastically formulated as shown in the following equation.

Here, I _{t, f} ^(m) represents a sound source index of a dominant sound source in each time frequency bin of the observation signal of the m-th microphone, and δ (·) represents a Dirac delta function. In the following description, the variable It _{, f} ^(m) is referred to as a dominant source index (DSI), and the subscript is omitted for simplicity.

Expression (3) represents that the observation signal ot _{, f} ^(m) in the m-th microphone is equivalent to the dominant sound source image in the microphone. Here, it should be noted that a different sound activity pattern, that is, a dominant sound source index DSI, is assigned to each microphone.

When the above probability model is used, the joint probability of the observation signals ot _{, f} ^(m) and I (dominant sound source index DSI) is derived as follows.

Θ ⁽ⁱ⁾ represents a parameter related to each sound source i, and θ represents a parameter related to all sound sources. That is, Expression (6) is a joint probability of the model parameter θ including the observation signal o _{t, f} ^(m) and I (dominant sound source index DSI). The following example will be described on the assumption that the sound source image x _{t, f} ^{(i, m) of} each sound source and the probability model of the observation signal are modeled as described above. In the following description, the above-described LogMax approximate model (formula (4)) is referred to as a “LogMax observation model” or an “observation signal probability model”.

[Concept of this invention]
The sound source separation method of the present invention makes it possible to estimate a different activity pattern for each of a plurality of microphones by paying attention to important parameters included in the sound source image x _{t, f} ^{(i, m)} .

  An important parameter characterizing the sound source separation method of the present invention is the dominant sound source index DSI. Since the dominant sound source index DSI indicates an activity pattern of each microphone of each sound source, if this parameter can be estimated, it is possible to directly estimate an activity pattern that is different for each microphone.

In addition to the dominant sound source index DSI, the time-invariant microphone position dependence / sound source time-invariant gain β _f ^{(i, m)} and the time-variant microphone independence / sound source logarithm that are implicitly supporting the parameter The power spectrum s _{t, f} ⁽ⁱ⁾ is used (see equation (1)).

The principle that the activity pattern can be estimated by using these parameters will be briefly described. For example, if a certain sound source arrives at the m-th microphone with a high SNR, the microphone position-dependent / sound source time-invariant gain β _f ^{(i, m),} which is a parameter corresponding to the SNR, tends to take a relatively high value. The sound source is observed as the dominant sound source under the LogMax observation model.

A signal that is positively observed as the dominant sound source in a certain time frequency bin makes it possible to estimate the logarithmic power spectrum of that sound source. On the other hand, when a certain sound source arrives at the m-th microphone with a low SNR, the microphone position-dependent / sound source time-invariant gain β _f ^{(i, m)} tends to take a relatively low value. Becomes a non-dominant sound source. Under the LogMax observation model, the spectrum of the non-dominant sound source is not observed explicitly, so the logarithmic power spectrum of the sound source is not estimated.

  As described above, in the present invention, a microphone observation signal having a high SNR, generally close to the sound source, is mainly used to estimate the logarithmic power spectrum of each sound source. As a result, it is possible to estimate different activity patterns for each microphone while effectively taking into account information from a plurality of microphones.

In a specific embodiment, the activity pattern is estimated using an expected value maximization method (EM algorithm) with the dominant sound source index DSI as a latent variable. In step E (expected value), the posterior probability relating to the dominant sound source index DSI is updated, and information about which sound source is dominant in which time frequency bin of which microphone is estimated. In M step (update), based on the posterior probability, the microphone position-dependent / sound source time-invariant gain β _f ^{(i, m)} , microphone-independent / sound source log power spectrum _{st, f} ⁽ⁱ⁾ and error Update the variance σ _{t, f} ^{(i, m)} of the term _{et, f} ^{(i, m)} .

  FIG. 1 shows a functional configuration example of a sound source separation device 100 of the present invention. The operation flow is shown in FIG. The sound source separation device 100 includes a microphone-specific sound source presence posterior probability estimation unit 10, a model parameter estimation unit 20, and an output sound estimation unit 30. The function of each unit of the sound source separation device 100 is realized by reading a predetermined program into a computer configured by, for example, a ROM, a RAM, and a CPU, and executing the program by the CPU.

The microphone-specific sound source existence posterior probability estimation unit 10 collects sound source signals emitted from a plurality of sound sources with a plurality of microphones and a plurality of channel observation signals ot _{, f} ^(m), and the plurality of microphones observed with each of the microphones. The sound source existence posterior probability ^ M _{t, f} ^{(i, m)} for each sound source i is estimated for each microphone m using a model of an observation signal that is assumed that the sound pressure of the signal from each of the sound sources i is different. (Step S10). Here, the model of the observed signal is that the signal ot _{, f} ^{(i, m)} observed by the m-th microphone comes from each of a plurality of sound sources and is observed by the m-th microphone. , A model (LogMax observation model, equation (4)) defined to be equivalent to the incoming signal having the maximum sound pressure. The model of the incoming signal is that the sound source image x _{t, f} ^{(i, m)} of the i th sound source observed by the m th microphone is the microphone independent / sound source logarithmic power spectrum s _{t, f} ⁽ⁱ⁾ , the microphone position-dependent / sound source time-invariant gain β _f ^{(i, m)} corresponding to the sound pressure of the signal arriving at the m th microphone from the i th sound source, and the m th This is a probability model defined by an error term _{et, f} ^{(i, m)} corresponding to the difference between the signal arriving at the microphone and the signal from the i-th sound source observed by the m-th microphone (Equation ( 1)).

The microphone-independent / sound source logarithmic power spectrum s _{t, f} ⁽ⁱ⁾ may be referred to as a clean audio signal from a sound source that does not depend on the microphone. The microphone position-dependent / sound source time-invariant gain β _f ^{(i, m)} is a value that varies depending on the sound source and the microphone position, and may be referred to as a transfer function. It should be noted that the notation such as ^ is a correct notation that is located immediately above the variable as shown in the drawings and equations.

The model parameter estimation unit 20 receives the observation signals o _{t, f} ^(m) of a plurality of channels and the sound source existence posterior probability ^ M _{t, f} ^{(i, m)} estimated by the microphone-specific sound source existence posterior probability estimation unit 10 as inputs. Then, the model parameter ^ θ ⁽ⁱ⁾ of the observation signal is estimated (step S20). The model parameter ^ θ ⁽ⁱ⁾ includes microphone independent / sound source logarithmic power spectrum s _{t, f} ⁽ⁱ⁾ , microphone position dependent / sound source time invariant gain β _f ^{(i, m)} , and error term _{et, f} ^{( i, m)} variance σ _{t, f} ^{(i, m)} .

The output sound estimation unit 30 includes a plurality of channel observation signals ot _{, f} ^(m) , a sound source existence posterior probability ^ M _{t, f} ^{(i, m)} estimated by the microphone-specific sound source existence posterior probability estimation unit 10, and a model. Using the model parameter ^ θ ⁽ⁱ⁾ estimated by the parameter estimation unit 20 as an input, the sound source image x _{t, f} ^{(i, m)} relating to each sound source i is estimated and output for each microphone m (step S30).

The sound source separation apparatus 100 that operates as described above uses the sound source existence posterior probability ^ M _{t, f} ^{(i, m)} estimated for each sound source i in each of the plurality of microphones m, and the sound source image for each sound source i. Since x _{t, f} ^{(i, m)} is estimated, sound source separation can be performed efficiently even in a distributed microphone array environment. Hereinafter, the operation of the sound source separation device 100 will be described in more detail.

The sound source separation device 100 effectively estimates the model parameter {circumflex over ⁽ θ ⁾ } ^{(i) on} the basis of the maximum posterior probability (MAP). In this embodiment, the dominant sound source index DSI is regarded as a latent variable, and model parameters ^ ⁽ⁱ⁾ = (s _{t, f} ⁽ⁱ⁾ , β _f ^{(i, m)} , σ _{t, f} ^{(i, m ))} ). In order to perform efficient maximum posterior probability parameter estimation, this embodiment repeatedly maximizes the following auxiliary functions using the EM algorithm.

Here, θ represents a prior estimated value of the model parameter, and ^ θ represents an estimated value of the model parameter. Further, p (x _{t, f} ^{(i, m)} ; θ ⁽ⁱ⁾ ) in the equation (7) can be calculated from the pre-estimated value θ of the model parameter as defined in the equation (2). . It is assumed that the prior estimated value θ is given in advance. That is, the auxiliary function Q (θ | ^ θ) described above has the simultaneous probability p (o _{t, f} ^(m ) between the observed signal o _{t, f} ^(m) and the prior estimation value of the model parameter including the dominant sound source index DSI. ⁾ , It _{, f} ^(m) = i; θ ⁽ⁱ⁾ ) multiplied by the weight corresponding to the sound source posterior probability ^ M _{t, f} ^{(i, m)} for all the observed signals It is a weighted sum. In the EM algorithm, the model parameter is updated so that the value of the auxiliary function is increased.

The sound source existence posterior probability ^ M _{t, f} ^{(i, m)} in each microphone m can be expressed by the following equation.

  Equation (7) cannot be maximized analytically due to the complexity of the second term. Therefore, in this embodiment, the auxiliary function is maximized efficiently using the Newton-Raphson method.

  FIG. 3 shows a functional configuration example of a sound source separation device 100 ′ using the EM algorithm and the Newton-Raphson method. In addition to the configuration of the sound source separation device 100, the sound source separation device 100 ′ further includes a storage unit 40 and an iterative processing unit 50. The model parameter estimation unit 20 includes a microphone position dependent / sound source time invariant gain estimating unit 201 and a microphone independent / sound source logarithmic power spectrum estimating unit 202.

  The parameter optimization procedure is performed by the microphone-specific sound source presence posterior probability estimation unit 10, the model parameter estimation unit 20, the storage unit 40, and the iterative processing unit 50. FIG. 4 shows an operation flow of the parameter optimization procedure.

In the storage unit 40, model parameters _{ circumflex over ⁽ θ ⁾ } ⁽ⁱ⁾ = (^ s _{t, f} ⁽ⁱ⁾ , ^ β _f ^{(i, m)} , ^ σ _{t, f} ^{(i, m)} )) The updated value is stored. The storage unit 40 may store only the updated model parameter {circumflex over ⁽ θ ⁾ } ⁽ⁱ⁾ , and the initial value θ may be previously given as a constant to each unit that requires the value.

The microphone-specific sound source existence posterior probability estimation unit 10 uses the observation signal ot , f (m) for each of the plurality of microphones and the model parameter ^ θ (i) = (^ s t, f (i ) , ^ Β f (i, m) , ^ σ t, f (i, m) ) as inputs, and for each microphone, the sound source existence posterior probability ^ M t, f (i, m) is calculated (step S10). That is, the microphone by the sound source exists posteriori probability estimation unit 10, the observed signals o t, f (m) and the model parameters ^ theta when the fitted model of the observation signal (i), the observed signal o t, f (m ) And the model parameter {circumflex over ( θ ) } (i) , the sound source existence posterior probability {circumflex over ( M ) } t, f (i, m) is calculated. This process corresponds to the E step of the EM algorithm.

The microphone position-dependent / sound source time-invariant gain estimation unit 201 uses the observation signal ot _{, f} ^(m) for each of the plurality of microphones and the sound source presence posterior probability ^ M _{t, f} calculated by the microphone-specific sound source presence posterior probability estimation unit 10. ^{(I, m)} and the microphone parameter independent sound source log power spectrum ^ s _{t, f} ⁽ⁱ⁾ of the model parameter _{ circumflex over ⁽ θ ⁾ _} ⁽ⁱ⁾ stored in the storage unit 40 as input, The time invariant gain ^ β _f ^{(i, m)} and the variance σ _{t, f} ^{(i, m)} are calculated, and the value of the parameter stored in the storage unit 40 is updated (step S201). In the following expression, when the condition o _{t, f} ^(m) > (^ s _{t, f} ⁽ⁱ⁾ + ^ β _f ^{(i, m)} ) is satisfied, ^ κ _{t, f} ^{(i, m )} = ^ M _{t, f} ^{(i, m),} and if not satisfied, ^ κ _{t, f} ^{(i, m)} = 1.

The microphone-independent / logarithmic power logarithmic power spectrum estimation means 202 is arranged such that the observed signal ot _{, f} ^(m) for each microphone m, the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit 40, and the sound source existence posterior probability for each microphone. Using the sound source existence posterior probability ^ M _{t, f} ^{(i, m)} calculated by the estimation unit 10 as an input, the clean sound signal _{st, f} ⁽ⁱ⁾ from the i-th sound source that is common to the plurality of microphones m. ⁾ Is calculated by the following equation, and the value of the parameter stored in the storage unit 40 is updated (step S202). The processing of steps S201 and S202 (step S20) corresponds to the M step of the EM algorithm.

Also, it can be seen that the update formulas of ^ s _{t, f} ⁽ⁱ⁾ and ^ β _f ^{(i, m)} are similar. The difference between these update formulas is in the averaging process, where _{{circumflex over} ⁽ s ⁾ _{} t, f} ⁽ⁱ⁾ is calculated as the average over the microphone number, while _{{circumflex over} ⁽ β ⁾ _f ^{(i, m)} is calculated as the average over the time index. .

The auxiliary function in equation (9) is a value obtained by adding the auxiliary function defined in equation (7) and the value calculated in equation (12) multiplied by the weight ρ. This is because if there is a sound source that does not dominate at all in a certain microphone (a sound source that is not positively observed under the LogMax observation model ⁾ , the optimal solution for the microphone position-dependent and sound source time-invariant gain ^ β _f ^{(i, m)} Becomes infinitesimal and the entire estimation process becomes unstable. As described above, the following normalization term (prior distribution) 203 is defined for the microphone-independent / sound source log power spectrum ^ s _{t, f} ⁽ⁱ⁾ , and added to the auxiliary function with the weight ρ, like this Problems can be avoided.

  The normalization term 203 may be stored in the storage unit 40 in advance, or may be provided as a constant inside the model parameter estimation unit 20 as shown in FIG.

As described above, in the model parameter estimation unit 20, the auxiliary function of Expression (7), that is, the observation signal o _{t, f} ^(m) and the current model parameter estimation value θ ⁽ⁱ⁾ are applied to the observation model. , The joint probability p (ot _{, f} ^(m) , It _{, f} ^(m) = i; of the observed signal o _{t, f} ^(m) and the model parameter estimate θ ⁽ⁱ⁾ including the dominant sound source index DSI; θ ⁽ⁱ⁾ ) is multiplied by the weight corresponding to the sound source posterior probability ^ M _{t, f} ^{(i, m)} , and the model parameter is set so that the weighted sum of all the observed signals is increased. (Mic position dependent / sound source time-invariant gain ^ β _f ^{(i, m)} , variance σ _{t, f} ^{(i, m)} and microphone independent / sound source log power spectrum ^ s _{t, f} ⁽ⁱ⁾ ) are updated ( Formulas (9) to (11)).

The iterative processing unit 50 repeats the E step and the M step until a predetermined criterion is satisfied (step S51). As the predetermined standard, for example, the Q function (auxiliary function) shown in the equation (7) calculated from the model parameter ^ θ before update and the sound source existence posterior probability ^ M _{t, f} ^{(i, m) for} each sound source When the difference between the value and the value of the Q function calculated from the updated model parameter and the sound source existence posterior probability ^ M _{t, f} ^{(i, m) for} each sound source is less than a predetermined threshold, There are a method for determining that the standard is satisfied, and a method for determining that the predetermined standard is satisfied when a predetermined number of repetitions is reached. The auxiliary function can be maximized by repeating the process.

When the predetermined criterion is satisfied, the output sound estimation unit 30 uses the observation signal ot _{, f} ^(m) for each of the plurality of microphones and the sound source existence posterior probability ^ M _t, calculated by the microphone-specific sound source existence posterior probability estimation unit 10 _{. Using f} ^{(i, m)} and the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit 40 as input, the i-th sound source image ^ x _{t, f} ^{(i, m)} in the m-th microphone is calculated. And output. When parameter estimation is performed using the EM algorithm, a sound source image ^ x _{t, f} ^{(i, m)} can be obtained by least square error estimation. The estimated sound source image ^ x _{t, f} ^{(i, m)} is expressed by the following equation.

[Evaluation experiment]
An evaluation experiment was conducted for the purpose of evaluating the performance of the sound source separation device 100 of the present invention. The experimental conditions were as follows.

  FIG. 5 shows the acoustic environment used for the simulation. The size of the room is 10 m (W) × 5 m (D) × 5 m (H), and the reverberation time is 100 ms. This acoustic environment is mirror image (Reference 1: JB Allen and DA Berkeley, “Image method for efficiently simulating small-room acoustics,” J. Acoust. Soc. Am., Vol. 65 (4), pp. 943-950. , 1979.).

  Four acoustic environments were simulated. The first acoustic environment and the second acoustic environment are assumed to be a situation in which three speakers sit at equal intervals in a circle with a radius of 80 cm and talk at the same time. The first acoustic environment is a situation where three microphones are arranged concentrically with a radius of 10 cm, and the second acoustic environment is a situation where the same microphone is arranged concentrically with a radius of 50 cm. In FIG. 3, the first acoustic environment and the second acoustic environment are in a state where there is no group of two speakers and a microphone.

  The 3rd acoustic environment and the 4th acoustic environment assumed the situation where two groups of three speakers and two speakers are talking in the same room. The third acoustic environment is a situation where five microphones are arranged concentrically with a radius of 10 cm, and the fourth acoustic environment is a situation where the same microphone is arranged concentrically with a radius of 50 cm.

  Three sound sources were separated in the first and second acoustic environments. In the third and fourth acoustic environments, five sound sources were separated. The conventional method compared with the present invention is the method shown in Non-Patent Document 1 that performs sound source separation using a soft mask, assuming a sound source activity pattern common to all microphones. In the conventional method, the microphone observation signal closest to each sound source is subjected to soft mask processing, and the separated signal is calculated.

  In the method of the present invention, the processing result of the conventional method is used as the initial value of the EM algorithm. The processing result of the conventional method was also used for the calculation of the normalization term shown in Equation (12). The normalization term weight ρ was set to ρ = 0.0001.

  The cepstrum distance was used as an evaluation index. The cepstrum distance is the distance between the comparison target signal and each sound source image in the microphone closest to each sound source. For evaluation speech, TIMIT (Reference 2: W. Fisher, GR Doddington, and KM Goudie-Marshall, “The DARPA speech recognition research database: specifications and status,” in Proc. DARPA workshop on Speech Recognition, 7986, pp. 96-99.), Randomly mixed speech was used to prepare 20 different mixed speech in each acoustic environment, and the result was calculated as an average value thereof.

  FIG. 6 shows the results of the evaluation experiment. The horizontal axis is the acoustic environment, and the vertical axis is the cepstrum distance (dB). The observation signal, the conventional method, and the cepstrum distance of the present invention are shown for each acoustic environment. Here, in order to calculate the cepstrum distance of the observation signal, the observation signal of the microphone closest to each speaker is used, which corresponds to the result of the microphone selection process when the nearest microphone is known.

  As a result of the first acoustic environment, the cepstrum distance is reduced even in the conventional method, but the present invention can further reduce the cepstrum distance. This is considered because the method of the present invention performs parameter optimum estimation in the logarithmic power spectrum region similar to the cepstrum region.

  In the second to fourth acoustic environments, the performance improvement by the conventional method cannot be confirmed. In the conventional method, performance is degraded on the cepstrum distance scale, and distortion is expected to increase due to over-suppression. The method of the present invention can effectively reduce the cepstrum distance in all acoustic environments. Thus, according to the sound source separation apparatus 100 of the present invention, it was confirmed that sound source separation was performed efficiently even in a distributed microphone array environment.

  When the processing means in the speech separation apparatus 100 described above is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  For the purpose of efficiently estimating the maximum posterior probability parameter, the sound source separation apparatus 100 ′ using the EM algorithm Newton-Raphson method has been described, but the present invention is not limited to this embodiment. For example, it is not necessary to use the EM algorithm to perform maximum posterior probability parameter estimation. Even if an all combination search method for searching all combinations is used, it is within the scope of the technical idea of the present invention.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (7)

It is assumed that the sound pressure of the signals from the multiple sound sources observed by each of the multiple microphones is different from the observation signal of the multiple channels obtained by collecting the sound source signals emitted from the multiple sound sources by the multiple microphones. A microphone-specific sound source presence posterior probability estimator that estimates the sound source presence posterior probability for each sound source for each microphone using the observed signal model;
A model parameter estimator for estimating the model parameters of the observation signal, using the observation signals of the plurality of channels and the sound source existence posterior probability as inputs;
An output sound estimator that estimates and outputs an incoming signal from each sound source for each of the microphones by using the observation signals of the plurality of channels, the sound source existence posterior probability, and the model parameter;
A sound source separation apparatus comprising: The sound source separation device according to claim 1,
The model of the observed signal is
A signal ot _{, f} ^(m) observed by the m-th microphone (where t is a time index and f is a frequency index) arrives from each of the plurality of sound sources and is received by the m-th microphone. It is a model that is defined to be equivalent to the incoming signal with the maximum sound pressure among the observed incoming signals,
The incoming signal model is
The incoming signal x _{t, f} ^{(i, m)} from the i th sound source observed by the m th microphone is
clean sound signal _{st, f} ^{(i) of the} i-th sound source,
a transfer function β _f ^{(i, m)} corresponding to the sound pressure of the signal arriving at the m th microphone from the i th sound source,
an error term _{et, f} ^{(i, m)} corresponding to the difference between the signal arriving at the mth microphone from the ith sound source and the signal from the ith sound source observed at the mth microphone;
Is a probability model defined by
The model parameters are the clean sound signal s _{t, f} ^{(i) of the} sound source, the transfer function β _f ^{(i, m)} and the variance σ _{t, f} ^{(i )} of the error term _{et, f} ^{(i, m). , M)}
A sound source separation device characterized by that. In the sound source separation device according to claim 2 ,
Furthermore, a storage unit and an iterative processing unit are provided,
The storage unit stores the model parameter ^ θ ⁽ⁱ⁾ of the observation signal,
The microphone-specific sound source existence posterior probability estimation unit receives the observation signal ot _{, f} ^(m) for each microphone m and the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit as inputs, and outputs the signal for each microphone m. When the observed signal o _{t, f} ^(m) and the model parameter ^ θ ⁽ⁱ⁾ are applied to the observed signal model, the observed signal o _{t, f} ^(m) and the observed signal model parameter ^ θ ^{(i )} And the sound source existence posterior probability ^ M _{t, f} ^{(i, m)} for each microphone m and sound source i,
The model parameter estimator includes the observation signal ot _{, f} ^(m) for each microphone m, the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit, and the sound source posterior probability ^ M _{t, f} ^(i, and ^m) as an input, the observed signal when the observed signal _{o t} for each said microphone ^{m, f} a model parameter ^ theta and ^{(i) (m)} was fitted to the model of the observed signal _{o t,} ^{f (m )} And the model parameter ^ θ ⁽ⁱ⁾ of the observed signal multiplied by the weight corresponding to the sound source posterior probability ^ M _{t, f} ^{(i, m)} to all the observed signals The transfer function β _f ^{(i, m)} and the variance σ _{t, f} ^{(i, m)} of the error term _{et, f} ^{(i, m)} stored in the storage unit are increased so that the weighted sum added with respect to ^m) and clean audio signal _{st, f} ⁽ⁱ⁾ Is a new one,
The iterative processing unit repeats the processing of the microphone-specific sound source presence posterior probability estimation unit and the model parameter estimation unit until a predetermined criterion is satisfied,
The output sound estimation unit receives the observation signals of the plurality of channels, the sound source existence posterior probability, and the parameter ^ θ ⁽ⁱ⁾ stored in the storage unit, and receives the incoming signal x _{t, f} ^{(i , M)}
A sound source separation device characterized by the above. It is assumed that the sound pressure of the signals from the multiple sound sources observed by each of the multiple microphones is different from the observation signal of the multiple channels obtained by collecting the sound source signals emitted from the multiple sound sources by the multiple microphones. A microphone-specific sound source presence posterior probability estimation process for estimating the sound source presence posterior probability for each sound source for each microphone using the observed signal model;
A model parameter estimation process for estimating the model parameters of the observation signal by using the observation signals of the plurality of channels and the sound source existence posterior probability as inputs,
An output sound estimation process for estimating and outputting an incoming signal from each sound source for each of the microphones by using the observation signals of the plurality of channels, the sound source existence posterior probability, and the model parameter;
A sound source separation method comprising: In the sound source separation method according to claim 4,
The model of the observed signal is
A signal ot _{, f} ^(m) observed by the m-th microphone (where t is a time index and f is a frequency index) arrives from each of the plurality of sound sources and is received by the m-th microphone. It is a model that is defined to be equivalent to the incoming signal with the maximum sound pressure among the observed incoming signals,
The incoming signal model is
The incoming signal x _{t, f} ^{(i, m)} from the i th sound source observed by the m th microphone is
clean sound signal _{st, f} ^{(i) of the} i-th sound source,
a transfer function β _f ^{(i, m)} corresponding to the sound pressure of the signal arriving at the m th microphone from the i th sound source,
an error term _{et, f} ^{(i, m)} corresponding to the difference between the signal arriving at the mth microphone from the ith sound source and the signal from the ith sound source observed at the mth microphone;
Is a probability model defined by
The model parameters are the clean sound signal s _{t, f} ^{(i) of the} sound source, the transfer function β _f ^{(i, m)} and the variance σ _{t, f} ^{(i )} of the error term _{et, f} ^{(i, m). , M)}
A sound source separation method characterized by the above. In the sound source separation method according to claim 5 ,
Furthermore, it has an iterative process,
The microphone-specific sound source existence posterior probability estimation process is performed by using the observation signal ot _{, f} ^(m) for each microphone m and the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit as input, and observing for each microphone m. The observed signal o _{t, f} ^(m) and the model parameter ^ θ ^{(i) of the} observed signal when the signal o _{t, f} ^(m) and the model parameter ^ θ ⁽ⁱ⁾ are applied to the model of the observed signal. And the sound source existence posterior probability ^ M _{t, f} ^{(i, m)} for each microphone m and sound source i,
The model parameter estimation process includes the observation signal ot _{, f} ^(m) for each microphone m, the model parameter ^ θ ⁽ⁱ⁾ stored in the storage unit, and the sound source existence posterior probability ^ M _{t, f} ^(i, and ^m) as an input, the observed signal when the observed signal _{o t} for each said microphone ^{m, f} a model parameter ^ theta and ^{(i) (m)} was fitted to the model of the observed signal _{o t,} ^{f (m )} And the model parameter ^ θ ⁽ⁱ⁾ of the observed signal multiplied by the weight corresponding to the sound source posterior probability ^ M _{t, f} ^{(i, m)} to all the observed signals The transfer function β _f ^{(i, m)} and the variance σ _{t, f} ^{(i, m)} of the error term _{et, f} ^{(i, m)} stored in the storage unit are increased so that the weighted sum added with respect to ^m) and the clean audio signal _{st, f} ⁽ⁱ⁾ To update,
The iterative process is to repeat the processes of the microphone-specific sound source presence posterior probability estimation process and the model parameter estimation process until a predetermined criterion is satisfied,
In the output sound estimation process, the received signals x _{t, f} ^{(i )} for each sound source i are input with the observation signals of the plurality of channels, the sound source existence posterior probability, and the parameter ^ θ ⁽ⁱ⁾ stored in the storage unit. ^{, M)} ,
A sound source separation method characterized by the above.   A program for processing the sound source separation method according to any one of claims 4 to 6 by a computer.

Images