JP5807914B2 - Acoustic signal analyzing apparatus, method, and program - Google Patents

Description

  The present invention relates to an acoustic signal analyzing apparatus, method, and program, and more particularly, to an acoustic signal analyzing apparatus, method, and program for separating time-series data of acoustic signals output from a plurality of microphones into signals of respective sound sources. .

  A technique that separates and extracts individual sound source components from the microphone input signal when both the sound source components and the transfer characteristics from the sound source to the microphone are unknown is called Blind Source Separation (BSS). In BSS, it is necessary to estimate the sound source signal and its mixing process only from the observed signal. Therefore, it is usually formulated as an optimization problem in which both unknown variables are estimated based on the criteria established by making some assumptions about the sound source. Is done. For example, when the number of observed signals is greater than or equal to the number of sound sources, a method called Independent Component Analysis (ICA), which estimates the maximum likelihood of the separation filter under the assumption that the sound source signal components follow a Gaussian distribution, is famous. (Non-Patent Document 1). ICA in the frequency domain is one of the effective approaches of BSS for mixed systems with time delays such as acoustic signals, but signal separation is performed for each frequency band, so the separation results for each band are separated for the same sound source. It was necessary to solve the so-called permutation problem.

Independent component analysis using vector-type variables (hereinafter, Independent Vector Analysis (IVA)), which has been proposed in recent years, is known as a separation technique that does not cause a permutation problem (Non-Patent Document 2). ). In IVA, the complex spectrum of each sound source for each frame is represented by a vector type variable Y _{k, τ} ^ = (Y _{k, τ, 1} , ..., Y _{k, τ, ω} , ..., Y _{k, τ, N} ) treated as ^T (where k is a sound source, τ is a time frame, ω is an index representing frequency), and its norm

A separation filter for each band is estimated under the assumption that follows a Gaussian distribution. In addition, a method for realizing IVA assuming a time-varying Gaussian distribution (a Gaussian distribution that allows dispersion to change with time) as a specific distribution of the dominant Gaussian distribution has been proposed (Non-patent Document 3).

The problem of IVA blind source separation is that the time-frequency representation of the observed signal X _Tω ^ = (X _1Tω , ..., X _MTω ) ^T

It is formulated as a problem of estimating the separation matrix W _ω ^ so that each element of is statistically independent. However, Y _τω ^ = (Y _1τω ,..., Y _kτω ,..., Y _Kτω ) ^T , which represents the time frequency component of each sound source.

A. Hyv¨arinen, J. Karhumen, and E. Oja, Independent Component Analysis, John Wiley & Sons, 2001. T. Kim, T. Eltoft and T. Lee, “Independent Vector Analysis: An Extension of ICA to Multivariate Components,” Proc. ICA, pp.165-172, 2006. Takuma Ono, Junki Ono, Shigeki Hiyama, “Sound Source Separation by Independent Vector Analysis Using Sound Source Activation as Advance Information,” Acoustical Society of Japan Autumn Meeting, pp.613-614, Sep. 2011.

  As described above, in IVA, it is assumed that the norm of the complex spectrum vectorized follows a Gaussian distribution, but as a Gaussian distribution, a time-varying Gaussian distribution (a Gaussian distribution that allows dispersion to change from frame to frame) This assumption assumes that the time envelope of power is the same for all frequencies for each sound source. However, in general, the target sound source may have a special property of harmonics such as music and voice. In this case, the amplitude envelope in the valley portion between the harmonic components does not necessarily coincide with other frequencies, while the dependency of power between the fundamental frequency and the harmonics can be an important clue for sound source separation. In addition, separation accuracy is limited for signals that do not meet the assumptions (sound source signals having harmonic characteristics such as pitch and period).

Also, in IVA, W _ω ^ is determined so that the assumption that the amplitude envelope of Y _Tω ^ is similar between all frequencies is satisfied as much as possible, so this assumption is large when the sound source has a harmonic structure. In some cases, components of other sound sources may be mixed in at locations corresponding to valleys of harmonic components, or permutation mismatch may occur. Therefore, if each sound source has a harmonic structure, it is better to assume a power spectrum density of the harmonic structure type for the sound source model. However, for this purpose, information on the fundamental frequency of the sound source is required, but information on the fundamental frequency of the sound source cannot usually be observed.

  The present invention has been made in view of the above circumstances, and an acoustic signal analyzing apparatus, method, and the like that can accurately separate sound source signals for each sound source from time series data of acoustic signals output from a plurality of microphones. And to provide a program.

In order to achieve the above object, an acoustic signal analyzing apparatus according to the present invention receives time series data of acoustic signals output from M (M is an integer of 2 or more) microphones m as input, and an observation time frequency component X _mτω (m microphones, tau is the time frame, omega is the index of the frequency.) and time-frequency analysis means for outputting a three-dimensional array X ^ with the elements, K number of sound sources k time-frequency component _Y kτω ( k is a sound source, τ is a time frame, and ω is a frequency index.) A power spectrum template λ ^(l) _ω having a harmonic structure in each time frame τ for each sound source k. (l is an index of the power spectral template.) is selected as a probability [pi ^(l) _Lkr dimensional array with the elements [pi ^, for each sound source k at each time frame τ power two-dimensional array sigma ^ with sigma ² _{k tau} elements, and for each frequency _{ω, X τω (= (X} 1τω, ···, X Mτω)) of the sound source signals by applying time-frequency component _Y τω ^ ( = (Y _1τω ,..., Y _Kτω )), the parameter initial value setting means for setting each initial value of the separation matrix W _ω ^, and all of (k, τ, ω, z _kτ ) in the combination, when z _Lkr and sigma ² _{k tau} is ^given, λ ^(zkτ) the probability density function of Y _Keitauomega represented by Gaussian distribution to distribute _{ω ·} σ ² _{k τ,} and [pi _Lkr is given Z _kτ probability at the time, the prior probability of the probability π ^(l) _kτ for all combinations of (k, τ, l), and the determinant of the separation matrix W _ω ^ for each frequency ω. Further, when the three-dimensional array X ^ is given, the three-dimensional array Π ^, the two-dimensional array Σ ^, and the frequency ω So as to maximize an objective function that represents the separation matrix W _omega ^ posterior probabilities, the three-dimensional array [pi ^, the two-dimensional array sigma ^, and parameter update means for updating the separation matrix W _omega ^ of each frequency omega And, based on the separation matrix W _ω ^ of each frequency ω and the three-dimensional array X ^, sound source signal estimated value update means for updating the three-dimensional array Y ^, until a predetermined end condition is satisfied, And an end determination unit that repeatedly performs the update by the parameter update unit and the update by the sound source signal estimated value update unit.

In the acoustic signal analysis method according to the present invention, the time-frequency analysis means receives time-series data of acoustic signals output from M (M is an integer of 2 or more) microphones m and inputs the observation time frequency component X _mτω ( m microphones, tau is the time frame, omega is the index of the frequency.) and the output of the three-dimensional array X ^ with the element, the parameter initial value setting means, K number of sound sources k time-frequency component _Y kτω ( k is a sound source, τ is a time frame, and ω is a frequency index.) A power spectrum template λ ^(l) _ω having a harmonic structure in each time frame τ for each sound source k. (Where l is an index of the power spectrum template ^{) a} three-dimensional array Π ^ whose elements are probabilities π ^(l) _kτ to be selected, and each time frame τ For the two-dimensional array Σ ^ having the power σ ² _{k τ} of the sound source k and each frequency ω, the time frequency of the sound source signal is caused to act on X _τω ^ (= (X _1τω ,..., X _Mτω )) the component _{Y τω ^ (= (Y 1τω} , ···, Y Kτω)) to set the initial value of each of the separation matrix W _omega ^ for obtaining the parameter updating means, (k, τ, ω, z kτ ) _And the probability density function of Y _kτω represented by a Gaussian distribution with variance of λ ^(zkτ) _ω · σ ² _{k τ} when z _kτ and σ ² _{k τ} are given, and π probability of z _{Lkr when Lkr} is given, (k, tau, l) the probability π ^(l) _kτ prior probabilities for all combinations, and the separation matrix W _omega ^ determinant for each frequency omega The three-dimensional array Π ^, the two-dimensional array Σ ^ given the three-dimensional array X ^, and Update the three-dimensional array Π ^, the two-dimensional array Σ ^, and the separation matrix W _ω ^ for each frequency ω so as to maximize the objective function representing the posterior probability of the separation matrix W _ω for each frequency ω. Then, the sound source signal estimated value updating means updates the three-dimensional array Y ^ based on the separation matrix _Wω ^ and the three-dimensional array X ^ of each frequency ω, and is determined in advance by the end determination means. Until the end condition is satisfied, the update by the parameter update unit and the update by the sound source signal estimated value update unit are repeated.

  The program according to the present invention is a program for causing a computer to function as each means of the acoustic signal analyzing apparatus.

As described above, according to the acoustic signal analysis device, method, and program of the present invention, the probability that each power spectrum template having a harmonic structure is selected for each sound source in each time frame for each frequency is an element. Using a three-dimensional array Π ^ having a three-dimensional array X ^ of observation time frequency components, a two-dimensional array Σ ^ having the power of each sound source as an element, and each frequency ω so as to maximize an objective function representing a separation matrix W _omega ^ posterior probabilities, three-dimensional array [pi ^, two-dimensional array sigma ^, and by repeatedly updating the separation matrix W _omega ^ of each frequency omega The effect is that the time-series data of the acoustic signals output from the plurality of microphones can be accurately separated into sound source signals for each sound source.

It is the schematic which shows the structure of the acoustic signal analyzer which concerns on embodiment of this invention. It is a figure which shows a power spectrum template. It is a flowchart which shows the content of the acoustic signal analysis process routine in the acoustic signal analyzer which concerns on embodiment of this invention. (A) The graph which shows the evaluation result of the separation performance in the room E2A by SDR, and (B) The graph which shows the evaluation result of the separation performance in the room JR2 by SDR.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, a generation model of a sound source in which a spectrum template is incorporated in a dispersion parameter in a time-varying Gaussian distribution is established, and which spectrum template should be selected in each frame for each sound source. That is, the separation filter for each band is estimated simultaneously.

<Principle of the invention>
First, the principle of the present invention will be described. First, a sound source generation model will be described.

The fundamental frequency in each frame is regarded as a latent variable, and a harmonic-structured power spectrum template is selected according to the latent variable (basic frequency index z _kT ), and the sound source frequency component is determined based on the power spectrum. Assume that the generation process is done. Thus, it is considered that a sound source having harmonics should be appropriately modeled as compared with the sound source model assumed in the conventional IVA.

l = 1,..., L are harmonic structure template indexes, and λ ^(l) _ω is a harmonic structure template of a specific fundamental frequency (see FIG. 2). From the above, assuming the Gauss property of the sound source, the sound source generation model with z _kT given is expressed by the following equation (3).

Here, N _C (•; μ, σ ² ) represents a complex Gaussian distribution having an average μ and a variance σ ² , and σ _kT ² is the power of the sound source k in the frame τ. On the other hand, if the probability of z _Lkr = l is selected and [pi _Lktau, latent variable z _Lkr is represented by discrete distributions shown in the following equation (4).

Further, for the purpose of inducing π _kτ to be sparse, the hyper prior distribution of π _kτ is assumed to be a Dirichlet distribution represented by the following equation (5).

  The problem of IVA in which the above sound source generation model is incorporated in the above equation (2) results in the problem of maximizing the objective function shown in the following equation (6).

However, X ^ = {X _mτω }, Π ^ = {π ^(l) _kτ }, Σ ^ = {σ ² _kτ }, W ^ = {W _ω }, Θ ^ = {Π, Σ, W} . T is the number of time frames. Note that “＾” attached to a symbol indicates that the symbol is a matrix, a multidimensional array, or a vector.

Although the global optimum solution of the objective function of the above equation (6) cannot be obtained analytically, a local solution can be efficiently searched by the auxiliary function method. Here, if an auxiliary variable γ _kτω with Σ _lγ ^(l) _kτω = 1 is used, an auxiliary function can be designed as in the following equation (7).

<Update auxiliary variables>
Original update equation for the auxiliary variables ^{_{Σ l γ (l) kτω =}} 1 constraint, obtained as the following equation (8) Solving ^{_{∂Q / ∂ γ (l) kτω}} = 0.

  Here, λ is a template and l represents an index of the template.

<Parameter update>
π The updating expressions ^(l) _Lkr and _{^{σ 2 kτ, ∂Q / ∂π (}} l) kτ = 0, the following by solving _{^{∂Q / ∂σ 2 kτ = 0 (}} 9) equation (10) It is obtained according to the formula.

W _ω ^ is updated line by line as in the following formulas (11) to (14).

Here, e _k ^ is a unit vector (= [0, ..., 1, ..., 0] ^T ) in which the k-th vector element is 1 and the other elements are 0.

<Update of estimated sound source signal value>
The estimation of the sound source signal is updated as shown in the following equation (15) using the updated separation matrix.

However, Y _{τ, ω} ^ = (Y _{1, τ, ω} , ..., Y _{K, τ, ω} ) ^T.

<System configuration>
Next, when the present invention is applied to an acoustic signal analyzer that analyzes acoustic signals obtained from M (M ≧ 2) microphones and separates them into known K (K <M) sound source signals The embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 1, the acoustic signal analysis apparatus according to the embodiment of the present invention is a computer that includes a CPU, a RAM, and a ROM that stores a program for executing an acoustic signal analysis processing routine described later. It is configured and functionally configured as follows.

  The acoustic signal analysis device 100 includes an input unit 10, a calculation unit 20, a storage unit 30, and an output unit 40.

The input unit 10 receives time-series data of acoustic signals (multi-channel signals) output from the M microphones. The storage unit 30 stores time-series data of acoustic signals input from the input unit 10. In addition, the storage unit 30 stores the result of each process described later, and also the harmonic structure template λ ^(l) _ω (l = 1,..., L, ω = 1,. , N) and the parameter β.

  The calculation unit 20 includes a time frequency analysis unit 21, an initial setting unit 22, an auxiliary variable update unit 23, a parameter update unit 24, a time frequency component estimation unit 25, an end determination unit 26, and a signal conversion unit 27. It has. The parameter update unit 24 includes a probability update unit 241, a time-varying gain update unit 242, and a separation matrix update unit 243.

The time frequency analysis unit 21 receives an observed acoustic signal as a time-series signal of each microphone, and receives a time frequency component (observation time frequency component) X _mτω (m = 1,..., M, ω = 1, .., N, τ = 1,..., T indicate indices corresponding to microphones, frequencies, and time frames, respectively.) A three-dimensional array X having (m, ω, τ) as elements (each) calculate. Further, the calculated time frequency component X _τω is stored in the storage unit 30. More specifically, the time-frequency analysis unit 21 performs time-frequency analysis on each microphone m by using time-series data of the acoustic signal of the microphone as an input and using a short-time Fourier transform (STFT). As a result, the time frequency component X _mτω is calculated, and a matrix (amplitude spectrogram) X ^ = (X _mτω ) _{M × N × T} storing the time frequency component X _mτω is output. The time frequency component X _mτω may be calculated using wavelet transform.

The initial setting unit 22 sets initial values of the parameters π ^(l) _kτ , σ ² _kτ , and W _ω ^ used in processing that will be described later. An appropriate value may be set as an initial value using a random number. The initial setting section 22, based on the W _omega ^ initial value and time-frequency component X _Emutauomega of, according to the above (2) to calculate the initial value of the separated signal Yτω ^.

Auxiliary variable update section 23, (l, k, τ, ω) for each of all combinations of, [pi stored in the storage unit _{^{30 (l) kτ, σ 2}} kτ, Y kτω, λ (l) ω Based on the above, the auxiliary variable γ ^(l) _kτω indicating the probability that the observation time frequency component X _mτω belongs to the power spectrum template λ (l) ω of each index 1 is _updated according to the above equation (8), and the storage unit 30 To store.

The probability update unit 241 uses the power spectrum template according to the above equation (9) based on γ ^(l) _kτω and β stored in the storage unit 30 for each combination of (l, k, τ). The probability π ^(l) _kτ for selecting λ ^(l) _ω is updated and stored in the storage unit 30.

The time-varying gain updating unit 242 performs the above (10 ⁾ based on γ ^(l) _kτω , Y _kτω , λ ^(l) _ω stored in the storage unit 30 for each combination of (k, τ). ), The power σ _kT ² of the sound source k in the frame τ is updated and stored in the storage unit 30.

The separation matrix updating unit 243 performs the above operation based on γ ^(l) _kτω , λ ^(l) _ω , and σ _kτ ² stored in the storage unit 30 for each of all combinations of (k, τ, ω). The parameter ~ σ _kτω ² is updated according to the equation (11) and stored in the storage unit 30. Further, the separation matrix updating unit 243 determines the intermediate parameter for each combination of (k, ω) based on X _τω and ~ σ _kτω ² stored in the storage unit 30 according to the above equation (12). V _kω is updated and stored in the storage unit 30.

Then, the separation matrix updating unit 243 uses the above formulas (13) and (14) based on W _ω ^ and V _kω stored in the storage unit 30 for all the combinations of (k, ω). Accordingly, w _kω ^ (weight vector having the weight of each microphone m for the frequency component ω of the acoustic signal of the sound source k as an element) is updated and stored in the storage unit 30.

The time frequency component estimator 25 calculates each sound source for each combination of (τ, ω) based on W _ω ^ and X _τω ^ stored in the storage 30 according to the above equation (15). The vector Y _τω ^ = (Y _1τω ,..., Y _kτω ,..., Y _Kτω ) ^T representing the time frequency component is updated and stored in the storage unit 30.

The end determination unit 26 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, the auxiliary variable update unit 23, the parameter update unit 24, and the time frequency component estimation unit Each process of 25 is repeated. When it is determined that the end condition is satisfied, the end determination unit 26 proceeds to processing by the signal conversion unit 27. The signal conversion unit 27 converts Y _τω ^ stored in the storage unit 30 into a sound source signal of each sound source, and the output unit 40 outputs the sound source signal.

  As the termination condition, it may be used that the number of repetitions s has reached a predetermined number S. The end condition is that the difference between the value of the objective function when using the s-1th parameter and the value of the objective function when using the sth parameter is smaller than a predetermined threshold. It may be used as

<Operation of acoustic signal analyzer>
Next, the operation of the acoustic signal analysis apparatus 100 according to the present embodiment will be described. First, time-series data of acoustic signals from each microphone is input to the acoustic signal analyzing apparatus 100 as a signal to be analyzed and stored in the storage unit 30. Then, in the acoustic signal analysis apparatus 100, an acoustic signal analysis processing routine shown in FIG. 3 is executed.

First, in step S101, an acoustic signal in each time frame τ is read from the storage unit 30 for each microphone, and a time frequency analysis using a short-time Fourier transform is performed on the acoustic signal. A three-dimensional array X ^ having the time frequency component X _mτω as an element of each (m, τ, ω) is generated and stored in the storage unit 30.

In step S102, using random numbers, initial values of parameters Θ ^ = {＾^, Σ ^, W ^} are set and stored in the storage unit 30, and the time frequency component Y _kτω of each sound source signal is _set. Is generated in each (k, τ, ω) element and stored in the storage unit 30.

Next, in step S103, based on the parameters Π ^, Σ ^, Y ^ set in step S102 or the parameters Π ^, Σ ^, Y ^ updated in steps S104, S105, and S107 described later, The auxiliary coefficient γ ^(l) _kτω is _calculated for each combination of (1, k, τ, ω) according to the equation (8) and stored in the storage unit 30.

In step S104, based on the updated auxiliary coefficient _γ ^(l) kτω in step S103, according to the above (9), the power spectral template lambda ^(l) the probability _{that ω} is selected π a ^(l) _Lkr The combination of each (l, k, τ) is calculated and stored in the storage unit 30.

In step S105, based on the parameter Y ^ set in step S102 or the parameter Y ^ updated in step S107 and the auxiliary coefficient γ ^(l) _kτω updated in step S103, the above equation (10) Accordingly, the power σ _kτ ² of the sound source k in the time frame τ is calculated for each combination (k, τ) and stored in the storage unit 30.

In the next step S106, the parameter W ^ set in step S102 or the parameter W ^ updated in the previous step S106, the three-dimensional array X ^ generated in step S101, and the update in step S105 are updated. Based on the parameter Σ ^ and the auxiliary coefficient γ ^(l) _kτω updated in step S103, w _kω is _set for each combination of (k, ω) according to the above equations (11) to (14). Calculate and store in the storage unit 30.

In step S107, based on the parameter W updated in step S106 and the three-dimensional array X ^ generated in step S101, the time frequency component Y _kτω of each sound source signal according to the above equation (15). Is calculated for each (k, τ, ω) element and stored in the storage unit 30.

  In the next step S108, it is determined whether or not the number of repetitions s has reached S as an end condition. If the number of repetitions s has not reached S, it is determined that the end condition is not satisfied. Then, the process returns to step S103, and the processes of steps S103 to S107 are repeated. On the other hand, when the number of repetitions s reaches S, it is determined that the end condition is satisfied, and in step S109, the sound source signal of each sound source is based on the three-dimensional array Y that is finally updated in step S107. Is output by the output unit 40, and the acoustic signal analysis processing routine is terminated.

<Experimental result>
Next, for the purpose of showing the usefulness of the technique according to the present embodiment, the result of an experiment by simulation using a single melodic instrument as a sound source will be described. The experimental conditions are shown in Table 1 below.

  Two types of pre-recorded room impulse responses with different reverberation times (references (S. Nakamura, K. Hiyane, F. Asano, T. Nishiura and T. Yamada, “Acoustical Sound Database in Real Environments for Sound Scene Understanding and Hands-Free Speech Recognition, “Proc. LREC, pp. 965-968, 2000.)) was convoluted with a single melodic instrument sound (trumpet, saxophone), respectively, to obtain an observation signal. In the room E2A (reverberation time T60 = 300 ms), the sound source comes from the direction of {−20 °, 0 °}, and in the room JR2 (T60 = 470 ms), the sound source comes from the direction of {−30 °, 30 °}.

The weight template used in the method according to this embodiment (hereinafter referred to as the proposed method) is a harmonic structure template in which 24 harmonic components are attenuated by 1 / √n for each semitone from 220 Hz (A3) A total of 25 types of templates with the same weight at frequency were used. As the initial value of the separation matrix of the proposed method, the value obtained by updating the separation matrix 40 times by independent vector analysis based on time-varying Gaussian distribution was used, and then the proposed method was repeated 40 times to obtain a restored signal. The proposed method has two types, π ^(l) _kT , which takes into account the Dirichlet distribution (Prop (dir)) and the method that makes the prior distribution uniform (Prop (flat)). In the experiment assuming the Dirichlet distribution, the utility of the prior distribution becomes smaller as the frame length increases, so p (π _kT ) is multiplied by the frequency bin number for the purpose of adjusting the effect. In addition, annealing was performed to gradually reduce the parameter β of p (π _kT ) from 1 to 0.95 during the 40 iterations of the proposed method. At this time, π ^(l) _kτ sometimes became negative during the update. In that case, it was replaced with a sufficiently small positive number.

  For comparison, the independent vector analysis method (IVA (TVG)) assuming a time-varying Gaussian distribution as a sound source generation model (IVA (lap)) using the independent vector analysis method assuming a Laplace distribution as a sound source generation model (above Non-patent document 2), method by independent component analysis (ICA) (reference (H. Sawada, R. Mukai, S. Araki and S. Makino, “A Robust Approach to the Permutation Problem of Frequency-domain Blind source Separation, ”Proc. ICASSP, pp. 381-384, 2003.)) was performed 80 times to update the separation matrix.

  Results of evaluation by SDR (see references (E. Vincent, R. Gribonval and C. F´evotte, “Performance Measurement in Blind Audio Source Separation,” IEEE Trans. ASLP, pp.1462-1469, 2006.)) Is shown in FIG. FIG. 4A shows the evaluation results of the separation performance in the room E2A (reverberation time T60 = 300 ms) by SDR, and FIG. 4B shows the evaluation results of the separation performance in the room JR2 (T60 = 470 ms) by SDR. In both environments, the proposed method was superior to the conventional method. In particular, in room E2A, an improvement of about 6 dB was observed by considering the Dirichlet prior distribution.

As described above, according to the acoustic signal analysis device according to the embodiment of the present invention, the probability that each power spectrum template having a harmonic structure at each frequency and each time frame is selected for each sound source is an element. Using a three-dimensional array Π ^ having a three-dimensional array X of observation time frequency components, a two-dimensional array Σ ^ having the power of each sound source as an element, and each frequency ω so as to maximize an objective function that represents the separation matrix W _omega ^ posterior probabilities, three-dimensional array [pi ^, two-dimensional array sigma ^, and by repeatedly updating the separation matrix W _omega ^ of each frequency omega, It is possible to accurately separate sound source signals for each sound source from time-series data of acoustic signals output from a plurality of microphones.

  In addition, a plurality of power spectrum templates with harmonic structures are prepared, and a sound source generation model in which the spectrum template is incorporated in the dispersion parameter in the time-varying Gaussian distribution is established. By simultaneously estimating which spectral template should be selected in each time frame and the separation filter of each band, the accuracy of sound source separation can be improved when the sound source signal has a harmonic structure. .

Also, the difference in the algorithm from Non-Patent Document 3 described above is that the parameters (1) γ ^(l) _kτω and π ^(l) _kT and their update formulas (formulas (8) and (9)) are new. added was enough, (2) power sigma ² _Lkr each sound source, the (10) equation as _γ ^(l) kTω be calculated using, (3) intermediate variables V _kW, (11 ) The power spectrum estimated value at each time calculated according to the equation ~ σ ² _kτω (the difference that can take different values for each frequency ω is an important difference from the prior art) as in the above equation (12) To be updated.

  In addition, since individual sound source signals mixed from acoustic signals acquired by a plurality of microphones can be separated, applications such as a hands-free video conference system and a system for automatically creating conference contents are expected.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above-described acoustic signal analysis apparatus has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 21 Time frequency analysis part 22 Initial setting part 23 Auxiliary variable update part 24 Parameter update part 25 Time frequency component estimation part 26 Termination determination part 27 Signal conversion part 30 Storage part 40 Output part 100 Acoustic signal analysis apparatus 241 Probability update unit 242 Time-varying gain update unit 243 Separation matrix update unit

Claims (5)

Using time series data of acoustic signals output from M (m is an integer of 2 or more) microphones m as input, an observation time frequency component X _mτω (m is a microphone, τ is a time frame, and ω is a frequency index. .), A time-frequency analysis means for outputting a three-dimensional array X ^ having elements as elements,
A time-frequency component Y _kτω of K sound sources k (k is a sound source, τ is a time frame, ω is an index of frequency), and a three-dimensional array Y ^ having elements as elements, and each sound source k is adjusted in each time frame τ. A power spectrum template λ ^(l) _ω having a wave structure (where l is an index of the power spectrum template) is selected as a three-dimensional array Π ^ having a probability π ^(l) _kτ as an element in each time frame τ The time frequency of the sound source signal by acting on X _τω (= (X _1τω ,..., X _Mτω )) about the two-dimensional array Σ ^ having the power σ ² _{k τ} of each sound source k and each frequency ω. Parameter initial value setting means for setting initial values of the separation matrix W _ω ^ for obtaining the component Y _τω (= (Y _1τω ,..., Y _Kτω ));
(K, τ, ω, z kτ) in all combinations of, when z _Lkr and sigma ² _{k tau} is given, represented by a Gaussian distribution to distribute ^{_{λ (zkτ) ω · σ 2}} k τ Y _Keitauomega probability density function probability of z _Lkr when, and [pi _Lkr given, (k, tau, l) said for all combinations of probability _π ^(l) kτ priors and the for each frequency ω The three-dimensional array Π ^, the two-dimensional array Σ ^, and the separation matrix W of each frequency ω when the three-dimensional array X ^ is given, expressed using a determinant of the separation matrix W _ω ^. so as to maximize an objective function that represents the posterior probability of _omega ^, and parameter updating means the three-dimensional arrangement [pi ^, the two-dimensional array sigma ^, and for updating the separation matrix W _omega ^ of each frequency omega,
Sound source signal estimated value updating means for updating the three-dimensional array Y ^ based on the separation matrix _Wω ^ and the three-dimensional array X ^ of each frequency ω;
An end determination unit that repeatedly performs update by the parameter update unit and update by the sound source signal estimated value update unit until a predetermined end condition is satisfied,
An acoustic signal analyzing apparatus including: The objective function is defined as an auxiliary variable γ ^(l) _kτω indicating the probability that the observation time frequency component X _mτω belongs to the power spectrum template λ ^(l) _ω of each index l for all combinations of (k, τ, ω). The auxiliary function used
The parameter update means includes
Said three-dimensional array [pi ^, two-dimensional array sigma ^, and based on the previous Kipa Lower spectral template _{^{λ (l) ω, (k}} , τ, ω, l) for each of all combinations of said auxiliary variable gamma ^{( l)} Auxiliary variable updating means for updating _kτω ,
Probability update means for updating the three-dimensional array Π ^ based on the auxiliary variable γ ^(l) _kτω ;
On the basis of the auxiliary variable _γ ^(l) kτω and before Kipa Lower spectral template λ _^(l) ω, the power updating means for updating the two-dimensional array sigma,
The auxiliary variable _γ ^(l) kτω, the two-dimensional array sigma ^, the three-dimensional array X ^, and based on the previous Kipa Lower spectral template λ ^(l) _ω, the separation matrix W _omega ^ of each frequency omega A separating matrix updating means for updating;
The acoustic signal analysis device according to claim 1, comprising: The Probability [pi ^(l) the prior distribution of _Lkr, acoustic signal analyzer according to claim 1 or 2, wherein was Direkure distribution. The time-frequency analysis means X _mτω (where m is a microphone, τ is a time frame, ω) with time-series data of acoustic signals output from M (m is an integer of 2 or more) microphones m as input. Is a frequency index.) And outputs a three-dimensional array X ^ with elements
The parameter initial value setting means, the time-frequency component Y _Keitauomega of the K sound source k (where k sound sources, tau is the time frame, omega is the index of the frequency.) Three-dimensional array having the elements Y ^, each sound source k A power spectrum template λ ^(l) _ω (where l is an index of the power spectrum template ⁾ having a harmonic structure in each time frame τ is selected as a three-dimensional array Π having a probability π ^(l) _kτ as an element ^ effect, two-dimensional array sigma ^ with power sigma ² _{k tau} for each sound source k at each time frame tau elements, and for each frequency _{ω, X τω (= (X} 1τω, ···, X Mτω)) to And setting each initial value of the separation matrix W _ω ^ for obtaining the time frequency component Y _τω (= (Y _1τω ,..., Y _Kτω )) of the sound source signal,
Gauss with variance λ ^(zkτ) _ω · σ ² _{k τ} when z _kτ and σ ² _{k τ} are given for all combinations of (k, τ, ω, z _kτ ) by the parameter updating means. probability of z _Lkr when the probability density function of the represented Y _Keitauomega in distribution, and [pi _Lkr given, (k, tau, l) the probabilities for all combinations of _π ^(l) kτ priors and Expressed using the determinant of the separation matrix W _ω ^ for each frequency ω, the three-dimensional array Π ^ given the three-dimensional array X ^, the two-dimensional array Σ ^, and each frequency ω I said to maximize the objective function representing a separation matrix W _omega ^ posterior probabilities, the three-dimensional array [pi ^, the two-dimensional array sigma ^, and updating the separation matrix W _omega ^ of each frequency omega of
Based on the separation matrix W _ω ^ and the three-dimensional array X ^ of each frequency ω, the three-dimensional array Y ^ is updated by the sound source signal estimated value update means,
An acoustic signal analysis method in which an update by the parameter update unit and an update by the sound source signal estimated value update unit are repeatedly performed by an end determination unit until a predetermined end condition is satisfied.   The program for functioning a computer as each means of the acoustic signal analyzer of any one of Claims 1-3.