JP5997007B2 - Sound source position estimation device - Google Patents

Description

  The present invention relates to a technique for estimating the arrival direction / position of an arbitrary sound source using observation signals collected by a plurality of microphones (hereinafter referred to as “sound source position estimation technique”). Note that the sound source direction estimation is defined as being included in the sound source position estimation.

  Various techniques have been studied for estimating the position of a sound source by using a phase / amplitude difference generated between observation signals picked up by a plurality of microphones (for example, microphone arrays). In the sound source position estimation technology so far, research has been carried out mainly with two policies: i) ingenuity regarding signal processing performed on collected observation signals, and ii) in arrangement of microphones.

[Conventional method i): Device for signal processing]
Typical signal processing includes: a) GCC-PHAT method (Generalized Cross Correlation with PHAse Transform method, see Non-Patent Document 1), b) MUSIC method (MUltiple SIgnal Classification method, see Non-Patent Document 2), c) Beamformer The law (see Non-Patent Document 3) is known.

[Conventional method ii): Device for array layout]
In the conventional method i), the position of the sound source has been estimated using signal processing. However, the performance varies greatly depending on the observation system (eg microphone arrangement). For example, a) Increasing the size of the microphone array can detect the difference between spherical and plane waves, thereby improving the accuracy of sound source position estimation, or b) arranging multiple microphone arrays at a distance. Thus, an approach for solving the problem of estimating the position of the sound source as a combination of direction estimation has been proposed (see Non-Patent Document 4). Basically, both systems have a policy of increasing the array size, and for the signal processing after observation, an arbitrary method may be used by taking the processing given in the conventional method i) as an example.

C. H. Knapp et al., “The generalized correlation method for estimation of timedelay”, IEEE Trans. ASSP, 1976, vol.24, no.4, pp. 320-327 R. O. Schmidt, “Multiple emitter location and signal parameter estimation”, IEEE Transactions on Antennas and Propagation, 1986, vol.34, no.3, pp.276-280 D. H. Johnson et al., "Array Signal Processing: Concepts and Techniques", USA, Prentice-Hall, 1993 Yusuke Hioka et al., “Enhanced sound collection in a specific 2D region using estimated sound source position information”, IEICE Technical Report (Applied Acoustics), 2008, Volume ： 108, Issue: 115, pp. 7-12

  Although it is sufficient that the microphone can be arranged so as to surround the sound source and the array size can be increased, the number M of microphones and the area where the microphone can be installed are usually limited. In that case, even if the signal processing as in the conventional method i) is devised, there is a possibility that the amount of information for estimating the sound source position in the observed signal is small and cannot be estimated. For example, it is assumed that a speaker sits side by side at a position about 5 to 10 m away using a sound collecting device for a video conference installed near the display. In that case, since the angle between the sound sources is narrow, it is difficult to identify the sound source.

  An object of the present invention is to provide a technique capable of estimating the position of a sound source even if the sound sources are arranged with a narrow angle difference and distance.

  In order to solve the above-described problem, according to the first aspect of the present invention, a sound source position estimation device is created from a plurality of microphones and a material capable of reflecting sound, and a plurality of assumptions of a sound source to be estimated Reflection means arranged in the vicinity of a plurality of microphones and a plurality of assumptions of the sound source to be estimated so that one or more reflected sounds can be collected in each microphone holon for the sound emitted from each of the positions. A transfer characteristic storage unit that stores transfer characteristics including the influence of reflected sound generated by the reflection means from a position to a plurality of microphones, an observation signal in a frequency domain obtained from a plurality of microphones, and a plurality of assumed positions. Using a corresponding transfer characteristic, a diffuse sensing unit that obtains an index indicating the high possibility that the estimation target sound source exists at the assumed position, and the estimation target sound source may exist And a sound source position estimating section for estimating a position corresponding to the index representing the higher the position of the sound source to be estimated.

  In the present invention, even if the sound sources are arranged with a narrow angle difference and distance by the diffusion sensing, there is an effect that the sound source position can be estimated.

The functional block diagram of the sound source position estimation apparatus which concerns on 1st embodiment. The figure which shows the processing flow of the sound source position estimation apparatus which concerns on 1st embodiment. FIG. 3A is a diagram showing a state where sound emitted from a sound source to be estimated is reflected by the reflecting means and arrives at the microphone in an isotropic direction, and FIG. 3B is a diagram showing a mirror image by the reflecting means of FIG. 3A. The figure which shows the example of a shape of a reflection means. The functional block diagram of the sound source position estimation apparatus which concerns on 2nd embodiment. The figure which shows the processing flow of the sound source position estimation apparatus which concerns on 2nd embodiment.

Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following description, the symbol “ ^- ” used in the text should be described immediately above the immediately preceding character, but it is described immediately after the character due to restrictions on text notation. In the formula, these symbols are written in their original positions. Further, the processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

In order to identify the position of the sound source and estimate the sound source position even in a situation where the sound sources are arranged at a narrow interval, control based on diffuse sensing (see Reference 1) is incorporated.
[Reference 1] K. Niwa et al., “Diffused sensing for sharp directivity microphone array”, ICASSP, 2012, pp. 225-228.

  In Reference 1, in an environment where powerful reflected sound comes randomly from all directions, such as in a tunnel or a cave, the target sound and Since it is possible to obtain the maximum information for spatially distinguishing other sound sources, it has become clear that it is possible to collect sound with narrow directivity. Diffuse sensing can be implemented, for example, by attaching a reflector around the microphone array.

For signal processing after sound collection, any method may be used. For example, the conventional methods such as b) MUSIC method and c) beamformer method can be used. In each method, a time difference and a transfer characteristic generated between microphones are prepared for each position r ⁻ _n where the sound source to be estimated can be assumed (hereinafter simply referred to as “assumed position”). However, in the conventional method, only the direct sound was modeled, so it was possible to calculate them easily by calculation.However, in the method based on diffuse sensing, the transfer characteristics were actually measured or the reflector It is necessary to prepare transfer characteristics including the effect of reflected sound, such as by simulating acoustic characteristics.

<Sound source position estimation apparatus 2 according to the first embodiment>
In the first embodiment, b) the sound source position is estimated by the MUSIC method. The MUSIC method estimates the positions of K sound sources k from the cues included in the observation signal using a larger number of microphones than the number of sound sources K existing in the sound field. Note that the number of sound sources K is estimated from a signal given or observed in advance. FIG. 1 is a functional block diagram of the sound source position estimating apparatus 2, and FIG.

  The sound source position estimation apparatus 2 includes M microphones 110-m, a reflection unit 200, an AD conversion unit 120, a frequency domain conversion unit 130, a transfer characteristic storage unit 210, a diffusion sensing unit 220, and a sound source position estimation. Part 160. The diffusion sensing unit 220 includes a noise space correlation matrix calculation unit 140 and a music spectrum calculation unit 150. However, M> K and m = 1, 2,.

The sound source position estimation apparatus 2 estimates K sound sources k using the analog observation signals x _m (i) collected by the M microphones 110-m, and estimates positions r ⁻ (τ) = [r _{^{- 1 (τ), ...,}} r - to output a _{K (τ)].} Details of the processing in each unit will be described below.

<Microphone 110-m and reflection means 200>
The reflecting means 200 is made of a material that can reflect sound, and M 1 can collect one or more reflected sounds at the microphone holon 110-m with respect to sounds emitted from the assumed positions r ⁻ _n. It arrange | positions in the vicinity of the microphone 110-m. However, when the number of assumed positions is N (≧ 1), n = 1, 2,. The M microphones 110-m collect sound emitted by the sound source that is the target of position estimation (s 3), and output the analog observation signal x _m (i) to the AD converter 120.

  In the present embodiment, diffusion sensing (see Reference 1) is implemented, and a technique for estimating the position of a sound source is realized even if sound sources to be estimated are arranged at narrow intervals. Note that diffusion sensing means “allowing space control that effectively uses a multi-channel sensor by observing a signal in a diffusion state”. Hereinafter, an observation signal that is reflected once or more and arrives at the microphone is referred to as a spread signal. The higher the number of reflections, the better.

  In the present embodiment, M microphones 110-m and reflecting means 200 are appropriately arranged so that the diffused signal can be picked up. For example, as described in Reference 1, the reflection means 200 is made of ABS resin (Acrylonitrile, butadiene (Butadiene), styrene (Styrene) copolymer synthetic resin) having a heat of 8 mm, and has a shape thereof. The octahedron with the tip cut off may be used, and 24 microphones may be arranged at the apexes inside. FIG. 3A shows a state in which the sound emitted from the sound source 100-k to be estimated is reflected by the reflecting means 200 and arrives at the microphone 110-m in the same direction, and FIG. 3B shows a mirror image by the reflecting means 200 of FIG. 3A. Show. The observation signal collected by the microphone 110-m in this way is a spread signal. The spread signal can be said to be a signal close to the spread state. For example, a reverberant sound when emitted in a tunnel or cave is close to a diffuse signal.

  As the number of reflections increases before the sound radiated from the sound source 100-k to be estimated is observed by the microphone 110-m, the observation signal becomes a diffuse signal and the sound source position estimation accuracy is improved (Reference Document 1). reference). Therefore, it is desirable that the material of the reflecting means 200 is a material that reflects without absorbing much sound. Moreover, it is desirable that the shape be a shape that increases the number of reflections. For example, the above-mentioned octahedron is cut off. Moreover, as shown to FIG. 4 (A) and (B), respectively, the shape which made the one surface of the dodecahedron and the icosahedron open may be sufficient, and it shows to FIG. 4 (C) and (D), respectively. In this way, the rhombus dodecahedron and the sphere may be provided with openings. Moreover, the shape which attached the horn etc. to the opening surface or the opening part may be sufficient.

<Transfer characteristic storage unit 210>
The transfer characteristic storage unit 210 stores in advance transfer characteristics a ⁻ (ω, r ⁻ _n ) including the influence of reflected sound generated by the reflection means 200 from the assumed position r ⁻ _n to M microphones 110 -m. (S1). However, the position of the microphone 110-m p ^- When ^{_{m, a - (ω, r}} - n) = [a 1 (ω, p - 1, r - n), ..., a M (ω, p - M , r ^- _n )] ^T.

Incidentally, transfer characteristics _{^{a - (ω, r - n}} ) is assumed position r ^- a transfer characteristic of the direct reach direct sound to the sound from the _n is M microphones 110-m, the sound is reflected by the reflector This is expressed as the sum of one or more reflected sounds that reach the M microphones 110-m.

Transfer characteristics _{^{a - (ω, r - n}} ) , for example, the sum of the steering vectors of the direct sound, the respective transfer characteristics of one or more reflected sound difference of arrival time is corrected for attenuation and the direct sound of the sound by reflection And In Reference 1, the transfer characteristics by the following formula _{^{a - (ω, r - n}} ) is obtained.

However, h- ⁽⁰⁾ (ω, r ^- _n ) is a direct sound steering vector, h- ^(d) (ω, r ^- _n ) (where 1≤d≤D) is a reflected sound steering vector, κ ^(d) (ω) is the reflection coefficient for the d-th reflected sound, p ^- _m ^(d) is the position of the d-th virtual microphone (mirror image) of the microphone 110-m, v is the speed of sound, and || p ⁻ _m ^(d) −r ⁻ _n || represents the distance from the sound source n to the d-th mirror image of the microphone 110-m. Further, the transfer characteristic a ⁻ (ω, r ⁻ _n ) may be obtained by actual measurement in an actual environment, or may be obtained by simulation using the acoustic characteristics of the reflector.

<AD Converter 120 and Frequency Domain Converter 130>
The AD conversion unit 120 receives M analog observation signals x _m (i) and converts them into digital observation signals x _m (t) (hereinafter also simply referred to as “observation signals x _m (t)”) (s5). ) And output to the frequency domain converter 130. However, i and t represent indexes of continuous time and discrete time, respectively.

Further, the frequency domain transform unit 130 receives M observation signals x _m (t), and each of the frequency domain observation signals X _m (ω, τ) (hereinafter simply referred to as “observation signal X _m (ω, τ)”). (S7) and output to the noise spatial correlation matrix calculation unit 140 in the diffusion sensing unit 220. However, ω and τ represent indexes of discrete frequency and frame time, respectively, and ω = 1, 2,. Note that the frequency domain representation of the observed signal picked up by the m-th microphone X _m (ω, τ) ^{and, X - (ω, τ)} = [X 1 (ω, τ), ..., X M (ω, τ)] ^{T. T} represents transposition.

<Diffusion sensing unit 220>
The diffusion sensing unit 220 receives Ω number of observation signals X ⁻ (ω, τ). The diffusion sensing unit 220 takes out N × Ω pieces of transfer characteristics a ⁻ (ω, r ⁻ _n ) from the transfer characteristic storage unit 210 in advance. Then, the diffusion sensing unit 220 uses the Ω observation signals X ⁻ (ω, τ) and N × Ω transfer characteristics a ⁻ (ω, r ⁻ _n ) to estimate the target position r ⁻ _n. An index representing the high possibility that the sound source is present is obtained (s8) and output to the sound source position estimating unit 160. Hereinafter, the processing content will be described in more detail.

(Noise spatial correlation matrix calculation unit 140)
The noise spatial correlation matrix calculation unit 140 receives Ω number of observation signals X ⁻ (ω, τ), and uses this value to calculate a noise spatial correlation matrix R ⁻ _N (ω, τ) for each frequency ω. (S9) and output to the music spectrum calculator 150.

The noise spatial correlation matrix calculation unit 140 first calculates a spatial correlation matrix R ⁻ (ω, τ) using Ω number of observation signals X ⁻ (ω, τ).

Here, H represents conjugate transposition. E [•] is an expected value operator, and can be replaced by, for example, temporal averaging. Next, in order to generate a spatial correlation matrix of the noise space, the spatial correlation matrix R ⁻ (ω, τ) is eigendecomposed.

^{Here, V - (ω, τ)} = [v - 1 (ω, τ), ..., v - M (ω, τ)] is M eigenvectors v ^- _m (ω, τ) configured eigenvectors It is a matrix. ^{Also, Λ - (ω, τ)} = diag ([Λ 1 (ω, τ), ..., Λ M (ω, τ)]) is, M eigenvalues Λ _m (ω, τ) eigenvalues composed of It is a matrix. The M eigenvalues Λ _m (ω, τ) are in the order of Λ ₁ (ω, τ) ≧... ≧ Λ _M (ω, τ) (see Reference 1). The first to Kth eigenvectors v ^- ₁ (ω, τ), ..., v ^- _K (ω, τ) contain components due to the sound source to be estimated, so from K + 1th to Mth , V ⁻ _M (ω, τ), there is only stationary noise in the space composed of eigenvectors v ⁻ _{K + 1} (ω, τ),. Using the property, a noise spatial correlation matrix R ⁻ _N (ω, τ) is generated.

That is, eigenvectors v ⁻ _{K + 1} (ω, τ),..., V ⁻ _M (ω, τ) and eigenvalues Λ _{K + 1} (ω, τ),. , Λ _M (ω, τ) and the noise spatial correlation matrix R ⁻ _N (ω, τ).

(Music spectrum calculator 150)
The music spectrum calculation unit 150 receives the spatial correlation matrix R ⁻ _N (ω, τ) of Ω noises, and this value and the N × Ω transfer characteristics a ⁻ (removed from the transfer characteristic storage unit 210. omega, r ^- from a _n), the following equation, for each frequency omega, and assumed position r ^- for each _n, music spectrum _{P MUSIC (ω, τ, r} - n) was calculated (s11), the sound source position estimation Output to the unit 160. However, n = 1, 2,..., N.

In the prior art, the transfer characteristic by modeling only the direct sound _{^{a - (ω, r - n}} ) but was not calculated, in the present embodiment, as described above, and calculated by modeling from the direct sound and the reflected sound ing. In the present embodiment, this music spectrum P _MUSIC (ω, τ, r ⁻ _n ) is used as an index indicating the high possibility that the estimation target sound source exists at the assumed position.

<Sound source position estimation unit 160>
The sound source position estimation unit 160 receives N × Ω music spectra P _MUSIC (ω, τ, r ⁻ _n ). Here, the music spectrum P _MUSIC (ω, τ, r ⁻ _n ) indicates that the greater the value, the higher the possibility that a sound source exists at the corresponding assumed position r ⁻ _n . Therefore, the sound source position estimation unit 160 extracts K positions corresponding to the large music spectrum P _MUSIC (ω, τ, r ⁻ _n ), estimates these positions as the sound source positions (s13), and estimates the position r ⁻ ( τ) = [r ^- ₁ (τ), ..., r ^- _K (τ)] is output.

For example, K items having the following large cost C _MUSIC (τ, r ^- _n ) are extracted, and K estimated positions r ^- _k (τ) corresponding to the cost C _MUSIC (τ, r ^- _k ) are output. To do.

<Effect>
By observing the sound by generating a diffuse reflection sound, the observation signal includes a clue for distinguishing the target sound from other sound sources even if the sound sources are arranged at narrow intervals. By performing signal processing in consideration of diffusive reflected sound (specifically, using transfer characteristics), it is possible to estimate the position of the sound source even if the sound sources are arranged at narrow intervals.

<Modification>
The AD conversion process (s5) and the frequency domain conversion process (s7) may be performed inside the microphone 110-m. In that case, the AD conversion unit 120 and the frequency domain conversion unit 130 are provided in the microphone 110-m.

<Sound source position estimation apparatus 3 according to the second embodiment>
Only parts different from the first embodiment will be described.

  In the second embodiment, c) the sound source position is estimated by the beamformer method. The beam former method is a method of estimating a sound source position by preparing a large number of beam formers and scanning a space.

  FIG. 5 is a functional block diagram of the sound source position estimating apparatus 3, and FIG.

  The sound source position estimation device 3 includes M microphones 110-m, a reflection unit 200, an AD conversion unit 120, a frequency domain conversion unit 130, a transfer characteristic storage unit 210, a diffusion sensing unit 330, and a sound source position estimation. Part 160. The diffusion sensing unit 330 includes a filter calculation unit 340 and a spatial spectrum calculation unit 350. The outline of the process (s32) in the diffusion sensing unit 330 is the same as the process (s8) in the diffusion sensing unit 220, and the details thereof are different. Details of the filter calculation unit 340 and the spatial spectrum calculation unit 350 different from those of the first embodiment will be described.

<Filter calculation unit 340>
Filter calculation unit 340, had been taken out from the transfer characteristic storage unit 210 N × Omega number of transfer characteristics _{^{a - (ω, r - n}} ) filter to scan the space from the _{^{w - (ω, r - n}} ) = ^{_{[W 1 (ω, r -}} n), ..., W M (ω, r - n)] and ^T, each frequency omega, (in other words, assuming the position r ^- each _n) each position of scanning computed (s33 ) And output to the spatial spectrum calculation unit 350. Although there are various filter design methods, in this embodiment, a) the delay sum method and b) the minimum variance method will be described.

In a) delay and sum method, as follows, assuming the position r ^- filter emphasizing cost sound in the _{^{n w - (ω, r -}} n) is designed.

b) The minimum variance distortion response method (MVDR method) is designed at the cost of minimizing noise energy while emphasizing the sound at the assumed position r ^- _n as follows: .

There are various other filter design methods, but the filter may be designed using any method.

The filter _{^{w - (ω, r - n}} ) , the transfer characteristics _{^{a - (ω, r - n}} ) After measurement of the may be calculated until the processing in the spatial spectrum calculation section 350.

<Spatial spectrum calculation unit 350>
The spatial spectrum calculation unit 350 receives N × Ω filters w ⁻ (ω, r ⁻ _n ) and Ω observation signals X ⁻ (ω, τ), and filters w ⁻ (ω , r ⁻ _n ) and the observation signal X ⁻ (ω, τ) are convolved to calculate a spatial spectrum P _BF (ω, τ, r ⁻ _n ) (s35) and output to the sound source position estimation unit 160.

In the present embodiment, this spatial spectrum P _BF (ω, τ, r ⁻ _n ) is used as an index representing the high possibility that the estimation target sound source is present at the assumed position. Therefore, the sound source position estimating section 160, music spectrum _{P MUSIC (ω, τ, r} - n) in place of the spatial spectrum _{P BF (ω, τ, r} - n) using, perform the same processing.

  With this configuration, the same effect as that of the first embodiment can be obtained.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
The above-described sound source position estimation apparatus can be functioned by a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a process procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

Claims (5)

There are multiple sound sources to be estimated, and the sound sources to be estimated are arranged with a narrow angle difference and distance,
A plurality of microphones;
A plurality of the plurality of reflected sounds can be collected in each microphone holon with respect to the sound generated from each of a plurality of assumed positions of the sound source to be estimated. Reflection means arranged in the vicinity of the microphone;
A transfer characteristic storage unit that stores transfer characteristics including the influence of reflected sound generated by the reflecting means from a plurality of assumed positions of a sound source to be estimated to the plurality of microphones;
Using the frequency domain observation signals obtained from the plurality of microphones and the transfer characteristics corresponding to the plurality of assumed positions, it is possible to increase the possibility that a sound source to be estimated exists at the assumed positions. A diffuse sensing unit for obtaining an index to represent,
It is seen including a sound source position estimating section for estimating a position at which the estimation target sound source corresponds to the index indicating the high possibility to be present as the estimated position of the target sound source, and
The plurality of assumed positions are arranged with a narrow angle difference and distance,
Sound source position estimation device.   The sound source position estimation apparatus according to claim 1,
  The reflection means has a shape that increases the number of reflections.
  Sound source position estimation device.   The sound source position estimation apparatus according to claim 2,
  The reflecting means is
  (i) a shape having one surface of the dodecahedron as an opening surface;
  (ii) a shape having one surface of an icosahedron as an opening surface;
  (iii) a shape having an opening in a rhomboid dodecahedron,
  (iv) a shape in which an opening is provided in a sphere,
  (v) a shape in which one of the vertices of the octahedron is cut out to provide an opening,
  Either
  Sound source position estimation device. The sound source position estimation device according to any one of claims 1 to 3 ,
The diffusion sensing unit is
A noise space in which a spatial correlation matrix is calculated using the observed signal, a spatial correlation matrix is calculated, and the spatial correlation matrix is eigendecomposed to obtain a noise spatial correlation matrix from eigenvectors and eigenvalues that do not include components derived from the sound source to be estimated A correlation matrix calculator,
A music spectrum calculation unit for calculating a music spectrum as the index from the transfer characteristics corresponding to the plurality of assumed positions and the spatial correlation matrix of the noise,
Sound source position estimation device. The sound source position estimation device according to any one of claims 1 to 3 ,
The diffusion sensing unit is
A filter calculation unit for calculating a filter for scanning the space using the transfer function;
A spatial spectrum calculation unit that convolves the filter and the observation signal and calculates a spatial spectrum as the index,
Sound source position estimation device.

Images