JP2018142917A - Sound source localization device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for simultaneous localization of multiple sound sources, even if the number of sound sources is unknown.SOLUTION: A sound source position estimation unit 25 estimates the position of each sound source k by estimating each parameter representing each distribution of a variable N, a variable ρ, a variable Z, a variable V, a variable λ, a variable ζ, and a variable γ, so as to minimize an objective function based on the variation inference method, with a function representing divergence indicating the difference between the exterior distribution p(Θ|Y) of an unknown variable Θ, including variables N and ρ representing the positions of multiple sound sources k when measurement data Y is given, a variable Z representing an indicator indicating the index of a sound source dominant at each time for each frequency, a variable V for determining a probability πwhen each sound source k is dominant, a variable λ representing decentralization of the observation time frequency components of each frequency for each of multiple directions, and variables ζ, γ representing the power of the time frequency components of each frequency of each sound source k and noise, and a variation function q(Θ) as an objective function.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a sound source localization device, method, and program, and more particularly, to a sound source localization device, method, and program for estimating the position of a sound source from an acoustic signal.

  Wave source localization has a wide range of applications such as radar and sonar. In particular, it is important to be able to quickly locate and track a moving wave source with a small array. Conventional methods for the source localization problem include multiple signal classicization (MUSIC) method, generalized cross-correlation methods with phase transform (GCC-PHAT) method, and methods based on source-constrained partial differential equations.

  Since the MUSIC method and GCC-PHAT method assume a plane wave for the sound source and use the arrival time difference between the sensors of each sound source as a key for localization, in general, a larger array size is advantageous. In addition, since both methods are based on statistics such as autocorrelation function and cross-correlation function between received signals of the sensor array, it is necessary to take a sufficiently long observation time width in order to localize the sound source with high accuracy. For this reason, these methods are not necessarily suitable for wave source localization using only a small array size and instantaneous observation. On the other hand, the method based on the partial differential equation of the wave source performs sound source localization based on the spatio-temporal partial differential equation of the acoustic signal that is established at each time. Theoretically, the source localization is performed only by instantaneous small region observation. Is possible.

  However, since this method is based on an equation that holds for a single wave source, it is vulnerable when the observed acoustic signal deviates from the partial differential equation, such as when there is noise or multiple point sources. Has drawbacks. Non-Patent Document 1 discloses a probability model of an array observation signal based on a wave source-constrained partial differential equation and a wave source localization algorithm based on it in order to enable expansion to the case where random noise and a plurality of point sound sources exist in this framework. Proposed.

Satoshi Suzuki and Hirokazu Kameoka, "Probabilistic modeling of acoustic signals based on wave source constrained difference equations and multiple sound source localization algorithms", Proceedings of the Acoustical Society of Japan, pp.615-618, March 2016

  In the method of Non-Patent Document 1, it is necessary to assume the number of sound sources, but the number of sound sources is often unknown in an actual environment. Therefore, it is desirable that sound source localization can be performed in accordance with the actual number of sound sources while adapting the complexity of the model without assuming the number of sound sources.

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a sound source localization apparatus, method, and program capable of simultaneously locating a plurality of sound sources even when the number of sound sources is unknown. And

In order to achieve the above object, a sound source localization apparatus according to the present invention is a sound source that estimates the position of each of a plurality of sound sources from an observation signal mixed with sound source signals from a plurality of sound sources input by a microphone array. A localization apparatus, for each of a plurality of directions, a spatial difference calculation unit that calculates a difference between the observation signals input by a pair of microphones arranged in the direction of the microphone array; Among them, the observation signal input by the reference microphone is input, and the observation time frequency component of each frequency is output, and the observation signal calculated for each of the plurality of directions by the spatial difference calculation unit is output. With the difference as an input, for each of the plurality of directions, a time frequency expansion unit that outputs an observation time frequency component of each frequency; and Based on the observation time frequency component of each frequency of the reference microphone and the observation time frequency component of each frequency for each of the plurality of directions output by the time frequency expansion unit, each frequency of the reference microphone A variable N and a variable ρ representing the positions of a plurality of sound sources k when given observation data Y composed of an observation time frequency component and an observation time frequency component of each frequency for each of the plurality of directions, and each frequency. A variable Z representing an indicator of the index of the sound source that becomes dominant at the time,
A variable V for determining the probability π _k that each sound source k becomes dominant, a variable λ representing the dispersion of the observation time frequency component of each frequency for each of the plurality of directions, and the time of each frequency of each sound source k and noise. A function representing a divergence representing a difference between a posterior distribution p (Θ | Y) of an unknown variable Θ including a variable ζ and a variable γ representing a power of a frequency component and a variable function q (Θ) is used as a variation function. Each parameter representing the distribution of each of the variable N, the variable ρ, the variable Z, the variable V, the variable λ, the variable ζ, and the variable γ is estimated so as to minimize the objective function based on an inference method. A parameter estimation unit, a parameter representing a distribution of the variable N representing the estimated position of each sound source k, and a sound source position estimating unit for estimating the position of each sound source k based on a parameter representing the distribution of the variable ρ. Consists of including

The sound source localization method according to the present invention is a sound source localization method in a sound source localization apparatus that estimates the position of each of the plurality of sound sources from an observation signal obtained by mixing sound source signals from a plurality of sound sources input by a microphone array. A spatial difference calculation unit for each of a plurality of directions, calculates a difference between the observation signals input by a pair of microphones arranged in the direction of the microphone array, and a time-frequency expansion unit In the microphone array, the observation signal input from a reference microphone is input, and an observation time frequency component of each frequency is output, and the spatial difference calculation unit calculates each of the plurality of directions. Using the observation signal difference as an input, the observation time frequency component of each frequency is output for each of the plurality of directions, and the parameters are output. The reference estimation unit outputs the reference time based on the observation time frequency component of each frequency of the reference microphone and the observation time frequency component of each frequency for each of the plurality of directions output by the time frequency expansion unit. A variable N and a variable ρ representing the positions of a plurality of sound sources k when given observation data Y composed of an observation time frequency component of each frequency of the microphone and an observation time frequency component of each frequency for each of the plurality of directions. , A variable Z representing an indicator of an index of a sound source that is dominant at each time for each frequency, a variable V for determining a probability π _k that each sound source k is dominant, and a frequency V for each of the plurality of directions. Variable λ representing the variance of the observed time frequency component, and variable ζ and variable representing the power of the time frequency component of each frequency of each sound source k and noise The objective function is minimized based on the variational inference method with the function representing the divergence representing the difference between the posterior distribution p (Θ | Y) of the unknown variable Θ including the variable function q (Θ) as the objective function. Thus, each parameter representing the distribution of each of the variable N, the variable ρ, the variable Z, the variable V, the variable λ, the variable ζ, and the variable γ is estimated, and the sound source position estimation unit The position of each sound source k is estimated based on a parameter representing the distribution of the variable N representing the position of the sound source k and a parameter representing the distribution of the variable ρ.

  The program according to the present invention is a program for causing a computer to function as each part of the sound source localization apparatus.

As described above, according to the sound source localization apparatus, method, and program of the present invention, from the observation time frequency component of each frequency of the reference microphone and the observation time frequency component of each frequency for each of the plurality of directions. When the observation data Y is given, a variable N and a variable ρ representing the positions of a plurality of sound sources k, a variable Z representing an index of a sound source dominant at each time for each frequency, and each sound source k A variable V for determining the probability π _k to be dominant, a variable λ representing the variance of the observed time frequency component of each frequency for each of the plurality of directions, and the power of the time frequency component of each frequency of each sound source k and noise A function representing the divergence representing the difference between the posterior distribution p (Θ | Y) of the unknown variable Θ including the variable ζ and the variable γ representing the variable function q (Θ) Each of the variables N, the variable ρ, the variable Z, the variable V, the variable λ, the variable ζ, and the variable γ is represented as a distribution so as to minimize the objective function based on the variational inference method. By estimating the parameters and estimating the position of each sound source k, it is possible to obtain an effect that a plurality of sound sources can be localized simultaneously even when the number of sound sources is unknown.

It is a figure which shows the spherical wave which arrives at the observation point r from a point sound source. It is a figure which shows an example of arrangement | positioning of a microphone array. It is the schematic which shows the structure of the sound source localization apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the sound source localization process routine in the sound source localization apparatus which concerns on embodiment of this invention. It is a figure which shows the sound source position and microphone position in experiment. It is a figure which shows the experimental result on the conditions of reverberation time 0.5 s and observation length 2779 ms. It is a figure which shows the experimental result on the conditions of reverberation time 0.5 s and observation length 1665 ms. It is a figure which shows the experimental result on the conditions of reverberation time 0.2 s and observation length 2779 ms. It is a figure which shows the experimental result on the conditions of reverberation time 0.2 s and observation length 1665 ms.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The technique proposed in the present invention is a signal processing technique for the purpose of estimating a wave source position from an acoustic signal.

<Outline of Embodiment of the Present Invention>
The embodiment of the present invention is a technology that enables wave source localization of a plurality of sound sources by small region / instantaneous observation, which has the advantages of the conventional method described above.

  In the embodiment of the present invention, the probability distribution of an acoustic signal based on the time-frequency domain representation of the wave source-constrained partial differential equation and the sparseness assumption of all sound sources including noise (the time of an acoustic signal in which multiple sound sources are mixed) In the frequency representation, based on the assumption that at most one sound source is dominant at each time frequency point), the source of each sound source is estimated using the variational inference algorithm to determine which sound source seems to be dominant at each time frequency point. Perform localization.

  Furthermore, in the embodiment of the present invention, it is possible to perform sound source localization according to the actual number of sound sources while adapting the complexity of the model without assuming the number of sound sources, with the idea inspired by the Dirichlet process mixture model. .

<Principle of Embodiment of the Present Invention>
Next, the principle of estimating the position of the sound source will be described.

<Sound source constrained partial differential equation>
As shown in FIG. 1, the position vector that becomes the reference of the observation point is

And the position vector of a single source

And Assuming that the wave source signal is g (t), the sound velocity is c, and spherical wave propagation from a single point source is assumed, the observed value at the observation point is

It is expressed. here,

It is. A unit vector from the observation point toward the wave source

Then,

Because

The spatial derivative of is

It becomes. Also,

Is the time derivative of

Therefore, substituting Equation (1) and Equation (8) into Equation (7) eliminates g,

Thus, an equation including only the observed signal and its time / space derivative can be established. However,

Is the distance from the observation point to the wave source. This equation is called a sound source constraint equation. As described above, the sound source constraint equation is a partial differential equation describing a unique relationship between the position of the sound source and the space field, which is formed by an arbitrary sound source signal waveform.

<Probability modeling of acoustic signal based on sound source constraint>
In order to approximate the spatial differentiation of the observation signal by a spatial difference, a microphone array shown in FIG. 2 is assumed below. However, the arrangement of the microphones is not limited as long as the spatial differentiation of the observation signal can be approximated by the spatial difference, and the following theory is not limited to the arrangement shown in FIG. In the case of the microphone array of FIG. 2, the signal f _{0, j} at the reference point and the spatial difference in each direction at each time t _j using seven microphones.



Can be obtained. Here, j represents the index of the sample time.

  Equation (9) is then

It can be expressed. Here, i = x, y, z, and n _x , n _y , and n _z are components of n in the x, y, and z directions, respectively. If we move the left side of equation (10) to the right side,

Is obtained. Here, let f _{0, j} , f _{x, j} , f _{y, j} , and f _{z, j} be signals obtained by windowing with a window function. Assuming that the influence of the two end points of the cut-out section can be ignored, Equation (11) can be expressed in the frequency domain.

It is expressed. Where F _{0, m} , F _{x, m} , F _{y, m} , F _{z, m} are discrete Fourier transforms of f _{0, j} , f _{x, j} , f _{y, j} , f _{z, j} , and m is Discrete frequency index.

  The right side of equation (12) is not always exactly 0 due to the error associated with the difference approximation. Therefore, the right side of equation (11)

Error variable like

These are normal random variables with mean 0 and independent of each other (random variables with complex normal distribution)

Assume that Further, each frequency component of the observation signal at the observation point is a normal random variable having an average of 0 and a variance of σ ² _{0, m} . this is,

Is equivalent to assuming.

here,

Vector and

Vector

And Equation (13) is

Can be written in the form of However,

And

Is

Given in. From equations (14) and (16),

Is average

, Variance covariance matrix

But

Complex normal distribution of

Follow.

So

Is

It is expressed. Therefore, from equation (22)

Probability density function

Get. From the above, observed spectrum and its spatial difference

Given the, the problem of localizing a single sound source is

Which results in a maximum likelihood estimation problem.

<Multiple sound source localization algorithms using the sparsity of sound sources>
Many real-world acoustic signals such as voice signals and musical sounds have sparse temporal frequency components. Therefore, even when a plurality of sound sources coexist at the same time, it can often be assumed that only one sound source is dominant at each time frequency point. The assumption of sparsity of the time frequency component of this sound source

Based on the probability modeling, it is possible to derive the probability distribution of the observed signal when there are a plurality of sound sources and when there is noise.

Assume that the time index of the signal cut frame is l, the sound source index is k = 0,..., K, k = 0 is noise, and k ≠ 0 is point sound source. Also, the position of the point sound source k

And Here, assuming the sparsity of the time-frequency components of all sound sources including noise, only the _{m, lth} sound source has non-zero power at frequency m and time l, and the power of other sound sources is assumed to be 0. To do. At this time, the time-frequency component of the observed signal under the given z _{m, l} and its spatial difference

The conditional probability density function of (observed signal) is

Given in. here,

It is. Also,

Is the variance-covariance matrix of the time-frequency component of noise, a normalized variance-covariance matrix model that depends only on frequency

Power of noise that also depends on time

Product of

It shall be represented by When diffusive noise is assumed

The setting method will be described later.

Is all unknown parameters

Represents.

Prior probability of

Then the observed signal

Probability density function of (

Is the likelihood function)

Can be written. From the above, the position of each sound source when multiple sound sources and noise exist

The problem of estimating

Under given

Which results in a maximum likelihood estimation problem. Although the global solution of this optimization problem cannot be solved analytically,

The stationary point can be searched by the Expectation-Maximization (EM) algorithm using as a latent variable.

<Non-parametric Bayes modeling>
In a real environment, the number of sound sources is often unknown. In the above formulation, it is assumed that the number of sound sources K is known. However, it is desirable that sound source localization can be performed according to the actual number of sound sources while adapting the complexity of the model without assuming the number of sound sources. Therefore, in the present embodiment, the generation model is extended to a Dirichlet process mixture model. k = 0 for noise,

Is the index of the point sound source,

Infinite dimensional discrete distribution

A random variable generated according to

Here, the probability π ₀ that the noise is dominant (z _{m, l} = 0) at the point (m, l) is expressed as a superparameter.

Generated according to the beta distribution of

And the probability π _k that the point sound source k ≠ 0 is dominant (z _{m, l} = k) is determined by the bar folding process.

And Generated by the above process

The expected value of tends to decrease exponentially as k increases, so that the sound source corresponding to large k has a lower probability of being active. Therefore, when the parameter is inferred from the observed signal, there is an effect of trying to explain the observed signal with a mixed model having the minimum number of sound source indexes. In the above generation model, all unknown variables Θ are as follows.

<Variation reasoning algorithm>
With the above generation modeling,

Can be written. The other prior distributions p (V), p (N), p (ρ), p (λ), p (ζ) and p (γ) are

Assuming that the observed signal

And unknown variable Θ

Can be specifically described. However,

Is the von Mise-Fisher distribution,

Is the real normal distribution,

Represents a gamma distribution. Posterior distribution of Θ

Is difficult to obtain analytically, but an approximate distribution can be obtained by iterative calculation based on the variational reasoning method. Variational reasoning is

Nonnegative variable function satisfying

The posterior distribution

Kullback-Leibler (KL) divergence between

The above equation (55) is used as an objective function.

The

Assuming that it can be approximated as

By iteratively minimizing the objective function of (55)

An approximate distribution can be obtained. Also,

With respect to the censored approximation

I do. This approximation does not mean that the complexity (number of sound sources) of the model is fixed, but means that the function space of q is limited to a certain range. Therefore,

Want to approximate as closely as possible

Increase

The range that can be taken should be widened.

  By substituting 0 for each q in Eq. (55), we get These are called variational posterior distribution update formulas.

However,

Is

of

The expected value for, and if X is a continuous variable

For discrete variables

Given in. Also,

Is

Represents the set of all elements in X except X. The derivation will be described in the next section. Each variational posterior update formula has the following form.

<Derivation of variational update formula>
<Bond distribution>
Specifically, log p (Θ, Y) can be written as follows using the generation model established above.

<Variable posterior distribution update formula for sound source direction N>

The term related to N in

And

Therefore, the variational posterior distribution update formula for N is

It becomes. Therefore, expected value

Is as follows.

<Variable posterior distribution update formula for sound source distance (reciprocal) ρ>

The term related to ρ in

And

Therefore, the variational posterior distribution update formula for ρ is

It becomes. Therefore, expected value

Is as follows.

<A variational posterior distribution update formula for active sound source index Z>

The term related to Z in

And

It becomes. However,

When k = 0,

When k ≠ 0

It is. Therefore, the variational posterior distribution update formula of Z is

It becomes. Therefore, expected value

Is as follows.

<Variable posterior distribution update formula for bar folding ratio V>

The term related to V in

And

Therefore, the variational posterior distribution update formula of V is

It becomes. Therefore, expected value

Is as follows.

However,

Is the digamma function.

<Error variable

Variational posterior distribution update formula for accuracy (reciprocal of variance) λ>

The term related to λ in

And

Therefore, the variational posterior distribution update formula for λ is

It becomes. Therefore, expected value

Is as follows.

<Variable posterior distribution update formula of sound source power (reciprocal number) ζ>

The term related to ζ in

And

Therefore, the variational posterior distribution update formula of ζ is

It becomes. Therefore, expected value

<Variable posterior distribution update formula of noise power (reciprocal of γ)>

The term related to γ in

And

Therefore, the variational posterior distribution update formula for γ is

It becomes. Therefore, expected value

Is as follows.

<Noise variance covariance matrix W>
Here, a setting example of the noise covariance matrix W _m when assuming the arrangement of seven microphones as shown in FIG. 2 will be described. here,

Fourier transform

And

and

The relationship is

To write

The variance-covariance matrix of

Then,

The variance-covariance matrix of

It becomes. Thus, for example, assuming spatially uncorrelated and equal power noise:

Is an identity matrix, so

Variance-covariance matrix of

The

And just put it.

  Within a certain area, a sound field with a uniform energy density and a distribution that can be regarded as having an equal probability of the flow of energy in all directions is called a diffuse sound field, and the sound field in a reverberant environment is approximated well. It is known to represent. In a diffuse sound field, the spatial correlation coefficient between two points depends only on the distance d,

Given in. Therefore, assuming diffuse noise, the example of an array geometry like Fig. 2

Variance-covariance matrix of

Is

It becomes. Using this,

Variance-covariance matrix of

The

And just put it. At this time,

Is

Is a diagonal matrix.

<Summary of variational inference algorithm>
The variational posterior distribution update formula for each variable is

The update formulas for the parameters of each distribution are as follows.

here

Is when k = 0

When k ≠ 0

It is. The expected value that appears in the update formula is given as follows.

<System configuration>
Next, an embodiment of the present invention will be described by taking as an example a case where the present invention is applied to a sound source localization apparatus that estimates the positions of a plurality of sound sources from acoustic signals input from a microphone array.

  As shown in FIG. 3, the sound source localization apparatus 100 according to the embodiment of the present invention is configured by a computer including a CPU, a RAM, and a ROM storing a program for executing a sound source localization processing routine. Functionally, it is configured as shown below.

  As shown in FIG. 3, the sound source localization apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90.

  The input unit 10 receives time-series data of an acoustic signal (hereinafter referred to as an observation signal) output from each microphone of the microphone array as shown in FIG. 2 and mixed with sound source signals from a plurality of sound sources.

  The calculation unit 20 includes a spatial difference calculation unit 22, a time frequency expansion unit 24, and a sound source position estimation unit 25.

The spatial difference calculation unit 22 acquires the observation signal f _{0, j} at the reference point microphone from each observation signal output from each microphone of the microphone array at each time t _j , and each direction x according to the following equation: , Y, z spatial differences f _{x, j} , f _{y, j} , f _{z, j} are calculated.

The time frequency expansion unit 24 calculates the observation time frequency component F _{0, m} of each frequency m from the observation signal f _{0, j at} each time t _j in the microphone of the reference point obtained by the spatial difference calculation unit 22. . In addition, the time frequency expansion unit 24 calculates the spatial differences f _{x, j} , _{fy, j} , f _{z, j in the} directions x, y, and z at each time t _j obtained by the spatial difference calculation unit 22. The observation time frequency components F _{x, m} , F _{y, m} and F _{z, m} of each frequency m are calculated. In the present embodiment, time frequency expansion such as short-time Fourier transform and wavelet transform is performed.

The sound source position estimation unit 25 is based on the observation frequency component y composed of the observation time frequency components F _{x, m} , F _{y, m} , F _{z, m} , F _{0, m of} each frequency m acquired by the time frequency expansion unit 24. Thus, a variable representing the position of a plurality of sound sources k when observation data Y comprising observation time frequency components F _{x, m} , F _{y, m} , F _{z, m} , F _{0, m of} each frequency m is given. N and a variable ρ, a variable Z representing an indicator of an index of a sound source that is dominant at each time for each frequency, a variable V for determining a probability π _k that each sound source k is dominant, and each of a plurality of directions A posteriori distribution p (Θ | Y) of an unknown variable Θ including a variable λ representing the variance of the observed time frequency components of each frequency, and a variable ζ and a variable γ representing the power of the time frequency components of each sound source k and noise. And the divergence representing the difference between the variable function q (Θ) Each parameter representing the distribution of each of the variable N, variable ρ, variable Z, variable V, variable λ, variable ζ, and variable γ so that the objective function is minimized based on the variational reasoning method with the function to be expressed as the objective function And the position of each sound source k is estimated.

  Specifically, the sound source position estimation unit 25 includes a variable update unit 28 and a convergence determination unit 30.

The variable updating unit 28 first sets each parameter representing the distribution of each of the variable N, variable ρ, variable Z, variable V, variable λ, variable ζ, and variable γ.

Set the initial value (hereinafter referred to as variational parameter).

In addition, the variable update unit 28 uses the observation data Y and the variation parameter.

In accordance with the above formula (132) to formula (146), the variational parameter
Update.

The convergence determination unit 30 determines whether or not a predetermined convergence determination condition is satisfied. When the convergence determination condition is not satisfied, the process of the variable update unit 28 is repeated. The convergence determination unit 30 is finally obtained when the convergence determination condition is satisfied.

The output unit 90 outputs the direction vector n ^(k) and the sound source distance R ^(k) of each sound source k as the estimation result of the position of each sound source k.

  As the convergence determination condition, it may be used that the number of iterations has reached a predetermined number. Note that the fact that the change rate of the parameter by one parameter update can be regarded as approximately 1 may be used as the convergence determination condition.

<Operation of sound source localization device>
Next, the operation of the sound source localization apparatus 100 according to the present embodiment will be described.

  When the input unit 10 receives time-series data of observation signals output from each microphone of the microphone array, the sound source localization apparatus 100 executes a sound source localization processing routine shown in FIG.

First, in step S120, the observation signal f _{0, j} at the reference point microphone is acquired at each time t _j from the time series data of the observation signal input from each microphone of the microphone array, and each direction x, y , Z spatial differences f _{x, j} , f _{y, j} , f _{z, j} are calculated.

In step S121, the observation time frequency component F _{0, m} of each frequency _m is calculated from the observation signal f _{0, j at} each time t _j in the microphone of the reference point obtained in step S120. In addition, the observation time frequency components F _{x, m} , F _{y, m of} each frequency m are obtained from the spatial differences f _{x, j} , f _{y, j} , f _{z, j} in each direction x, y, z at each time t _j. , F _{z, m} is calculated.

In step S122, the variational parameter

Set the initial value of.

In step S124, the observation data Y calculated in step S121 and the initial value or last updated
In accordance with the above formula (132) to formula (146), the variational parameter
Update.

  In step S125, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S124. On the other hand, if the convergence determination condition is satisfied, the process proceeds to step S126.

In step S126, it was finally obtained in step S124.

, The direction vector n ^(k) and the sound source distance R ^(k) of each sound source k are output by the output unit 90 as the position estimation result of each sound source k, and the sound source localization processing routine is terminated.

<Experiment>
In order to verify the performance of the proposed method, we performed numerical simulations of multiple sound source localization in a reverberant environment. This time, we used a two-dimensional model in the x and y directions. The room size is 6 m x 10 m x 4 m, and seven microphones are arranged in the center at intervals of 0.03 m as shown in Fig. 2 above. The wall reflection coefficients were 0: 7308 and 0: 4566 (the reverberation times according to the Sabine reverberation formula correspond to 0.5 s and 0.2 s, respectively). The sampling frequency of the microphone was 32 kHz, the short-time Fourier transform frame length was 64 ms (overlap was 32 ms), and the total length of the observation signal was 2779 ms and 1665 ms. The number of sound sources was 3, and the positions of the sound sources were (1, 0, 0) m, (-0.5, 0.87, 0) m, and (-0.5, -0.87, 0) m from the center of the room (see Fig. 5). ). In FIG. 5, the cross mark indicates the sound source position, and the round mark indicates the center position of the microphone array. As the sound source signal, a speech speed variation type speech database (SRV-DB) was used. As the proposed method, the method based on variational reasoning (VBEM), the method of this embodiment (VBEM), the method based on the EM algorithm assuming the correct number of sound sources (3) (EM1), and the number of incorrect sound sources (6) are assumed. The EM algorithm method (EM2) was evaluated and compared with the conventional MUSIC method. The EM algorithm assumes stationary noise.

It was. In any method, the presence and direction of the sound source are estimated based on the detection threshold, and if there is an error in the angle with the true sound source direction within ± τ, it is correct, otherwise it is a drop error and the detected sound source The F scale was calculated by misinserting the direction that does not belong to any true sound source direction. For each τ, plots of the highest F scale when the detection threshold is changed are shown in FIGS. FIG. 6 shows the localization accuracy of each method under the conditions of a reverberation time of 0.5 s and an observation length of 2779 ms. FIG. 7 shows the localization accuracy of each method under the conditions of a reverberation time of 0.5 s and an observation length of 1665 ms. FIG. 8 shows the localization accuracy of each method under the conditions of a reverberation time of 0.2 s and an observation length of 2779 ms. FIG. 9 shows the localization accuracy of each method under the conditions of a reverberation time of 0.2 s and an observation length of 1665 ms. In many cases, it was confirmed that the method based on variational reasoning, which is the proposed method, achieved a higher precision localization than other methods.

As described above, according to the sound source localization apparatus according to the present embodiment, the observation data Y including the observation time frequency component of each frequency of the reference microphone and the observation time frequency component of each frequency in each of a plurality of directions. , A variable N representing the position of a plurality of sound sources k and a variable ρ, a variable Z representing an index of a sound source that is dominant at each time for each frequency, and each sound source k being dominant. A variable V for determining the probability π _k , a variable λ representing the variance of the observed time frequency component of each frequency in each of a plurality of directions, and a variable ζ representing the power of the time frequency component of each frequency of each sound source k and noise Based on the variational reasoning method, the function representing the divergence representing the difference between the posterior distribution p (Θ | Y) of the unknown variable Θ including the variable γ and the variable function q (Θ) is the objective function. In order to minimize the objective function, the parameters representing the distributions of the variable N, variable ρ, variable Z, variable V, variable λ, variable ζ, and variable γ are estimated, and the position of each sound source k is estimated. Thus, even when the number of sound sources is unknown, a plurality of sound sources can be localized simultaneously.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the sound source localization apparatus described above has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 22 Spatial difference calculation part 24 Time frequency expansion | deployment part 25 Sound source position estimation part 28 Variable update part 30 Convergence determination part 90 Output part 100 Sound source localization apparatus

Claims (7)

A sound source localization device that estimates the position of each of the plurality of sound sources from an observation signal mixed with sound source signals from a plurality of sound sources input by a microphone array,
For each of a plurality of directions, a spatial difference calculation unit that calculates a difference between the observation signals input by a pair of microphones arranged in the direction in the microphone array;
Of the microphone array, the observation signal input from a reference microphone is input, and an observation time frequency component of each frequency is output, and the spatial difference calculation unit calculates each of the plurality of directions. With the difference between the observation signals as an input, a time frequency expansion unit that outputs an observation time frequency component of each frequency for each of the plurality of directions;
Based on the observation time frequency component of each frequency of the reference microphone output by the time frequency expansion unit, and the observation time frequency component of each frequency for each of the plurality of directions,
A variable N representing the position of a plurality of sound sources k when observation data Y comprising observation time frequency components of each frequency of the reference microphone and observation time frequency components of each frequency for each of the plurality of directions is given. And a variable ρ, a variable Z representing an indicator of an index of a sound source that is dominant at each time for each frequency, a variable V for determining a probability π _k that each sound source k is dominant, and each of the plurality of directions A posteriori distribution p (Θ | Y) of an unknown variable Θ including a variable λ representing the variance of the observed time frequency components of each frequency, and a variable ζ and a variable γ representing the power of the time frequency components of each sound source k and noise. And the variable N, so that the objective function is minimized based on the variational inference method, with the function representing the divergence representing the difference between the variable function q (Θ) and the objective function as the objective function. Serial variables [rho, and the variables Z, the variable V, the variable lambda, the variable zeta, parameter estimation unit for estimating the parameters representing each of the distribution of the variable gamma,
A sound source position estimator for estimating the position of each sound source k based on a parameter representing the distribution of the variable N representing the estimated position of each sound source k and a parameter representing the distribution of the variable ρ;
Sound source localization device including The sound source localization apparatus according to claim 1, wherein the probability π _k is determined so as to decrease exponentially as k increases. The variable function q (Θ) is approximated by q (N) q (ρ) q (Z) q (V) q (λ) q (ζ) q (γ),
The number of sound source truncations K ^* is determined in advance, and z _{ω, l} is an indicator indicating the index of the sound source that is dominant at each time l with respect to the frequency m,
3. The sound source localization apparatus according to claim 1, wherein k ( _{zω, l} = k ′) = 0 is set for k ′ that is equal to or greater than K ^* + 1. A sound source localization method in a sound source localization device that estimates the position of each of the plurality of sound sources from an observation signal mixed with sound source signals from a plurality of sound sources input by a microphone array,
A spatial difference calculation unit calculates, for each of a plurality of directions, a difference between the observation signals input by a pair of microphones arranged in the direction in the microphone array,
The time frequency expansion unit receives the observation signal input from a reference microphone in the microphone array and outputs an observation time frequency component of each frequency, and the spatial difference calculation unit outputs each of the plurality of directions. With respect to each of the plurality of directions, an observation time frequency component of each frequency is output as a difference between the observation signals calculated for
Based on the observation time frequency component of each frequency of the reference microphone and the observation time frequency component of each frequency for each of the plurality of directions, the parameter estimation unit is output by the time frequency expansion unit,
A variable N representing the position of a plurality of sound sources k when observation data Y comprising observation time frequency components of each frequency of the reference microphone and observation time frequency components of each frequency for each of the plurality of directions is given. And a variable ρ, a variable Z representing an indicator of an index of a sound source that is dominant at each time for each frequency, a variable V for determining a probability π _k that each sound source k is dominant, and each of the plurality of directions A posteriori distribution p (Θ | Y) of an unknown variable Θ including a variable λ representing the variance of the observed time frequency components of each frequency, and a variable ζ and a variable γ representing the power of the time frequency components of each sound source k and noise. And the variable N, so that the objective function is minimized based on the variational inference method, with the function representing the divergence representing the difference between the variable function q (Θ) and the objective function as the objective function. Serial variable [rho, the variables Z, the variable V, the variable lambda, the variable zeta, estimates the parameters representing each of the distribution of the variable gamma,
A sound source localization method in which a sound source position estimation unit estimates a position of each sound source k based on a parameter representing a distribution of a variable N representing the estimated position of each sound source k and a parameter representing a distribution of a variable ρ. The sound source localization method according to claim 4, wherein the probability π _k is determined so as to decrease exponentially as k increases. The variable function q (Θ) is approximated by q (N) q (ρ) q (Z) q (V) q (λ) q (ζ) q (γ),
The number of sound source truncations K ^* is determined in advance, and z _{ω, l} is an indicator indicating the index of the sound source that is dominant at each time l with respect to the frequency m,
6. The sound source localization method according to claim 4, wherein k ( _{zω, l} = k ′) = 0 is set for k ′ that is equal to or greater than K ^* + 1.   The program for functioning a computer as each part of the sound source localization apparatus of any one of Claims 1-3.