JP2010206393A - Speech direction estimation device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech direction estimation device which does not request to arrange many microphones so as to enclose a speaker, and can appropriately estimate the speech direction even under environment in which a reverberation time is long. <P>SOLUTION: The plurality of microphones pick up a sound signal originated by the speaker around a microphone array which is composed of the plurality of microphones. A correlation matrix which represents correlation between voice signals each picked up by each microphone, is created, and it is estimated that the speaker has spoken from what direction to the microphone array from an eigenvector obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for estimating the utterance direction of a speaker from an audio signal input to a microphone.

  A system for exchanging voice information such as a telephone or a voice conference terminal is generally called a voice communication system. In the video conference system, the video is added to the audio information and presented, so that the situation of the place is easily transmitted, but in the audio communication system, it is difficult to grasp the situation of the other party. One of the information on the other party's situation is utterance direction information. By receiving this information from the other party, it is possible to grasp the direction in which the speaker is speaking and to facilitate communication.

  Conventional techniques for estimating such speech direction information are disclosed in Non-Patent Documents 1 and 2 and the like, and a configuration example is shown in FIG. The speech direction estimation apparatus 10 in this configuration example estimates speech direction information as follows.

(i) The voice from the speaker 1 is picked up using M (M is an integer of 2 or more) microphones 11-1,. The collected analog signal is converted by the AD converter 12 into a digital signal vX (t) = [X ₁ (t),..., X _M (t)] ^T. Here, t represents an index of discrete time.

(ii) The frequency domain transform unit 13 receives a set (frame) of the digital signals composed of a plurality of samples as an input, and performs frequency domain signal vX (ω, n) = [X ₁ (ω, n) by fast Fourier transform or the like. _{, ···, X M (ω,} n)] to convert to ^T. Here, ω represents a frequency index, and the total number of frequency indexes is Ω. N represents the index of the frame.

(iii) In the fixed beamformer design unit 14, the fixed beamformer vG (ω, r, θ) = [G ₁ (ω, r, θ),..., G _M ( ω, r, θ)] ^T is designed. G _i (ω, r, θ) is a coefficient that is multiplied by the frequency component X _i (ω, n) of the i-th microphone in order to emphasize / suppress the sound source at the speaker position r and the speech direction θ.

In designing, acoustic propagation characteristics between the sound source and the microphone vH (ω, r, θ) = [H ₁ (ω, r, θ),..., H _{M for} each predetermined speaker position and direction. (ω, r, θ)] ^T is obtained using a simulation value or an actual measurement value. Here, H _i (ω, r, θ) represents acoustic propagation characteristics between the sound source at the speaker position r and the speech direction θ and the i-th microphone.

  The fixed beamformer vG (ω, r, θ) is designed as a value satisfying the expressions (1) and (2) representing the relationship with the acoustic propagation characteristics.

vH (ω, r _T , θ _T ) ^H · vG (ω, r _T , θ _T ) = 1 (1)
vH (ω, r _U , θ _U ) ^H · vG (ω, r _T , θ _T ) = 0 (2)
Expressions (1) and (2) emphasize the output power of the speaker position r _T and the speech direction θ _T , and the fixed beamformer so as to suppress the output power of the other speaker positions r _U and the speech direction θ _U. It shows that vG (ω, r, θ) is designed.

(iv) In the product-sum calculation unit 15, the frequency domain signal vX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T and the fixed beamformer vG (ω , r, θ) = [G 1 (ω, r, θ), ···, G M (ω, r, θ)] as input ^T, each frequency ω, speaker position r, every utterance direction θ Multiplying the frequency component X _i (ω, n) corresponding to each microphone by the coefficient G _i (ω, r, θ) of the fixed beamformer, and adding the obtained M components, the output Y (ω, n, r, θ) is calculated. This calculation is synonymous with calculating Y (ω, n, r, θ) = vG (ω, r, θ) ^H · vX (ω, n).

(v) The power calculator 16 calculates and outputs power | Y (ω, n, r, θ) | ² from the output Y (ω, n, r, θ) from the product-sum calculator 15.

(vi) The frequency averaging processing unit 17 averages the power | Y (ω, n, r, θ) | ² output from the power calculation unit 16 by frequency, and aY (n, r, θ) is obtained. obtain. In this calculation, if F ₀ is defined as the frequency index used in the averaging process, and | F ₀ | is defined as the total number of frequency indexes,

を計算することと同義である。なお、Ｆ_０はΩ≧|Ｆ_０|を満たす。

Is equivalent to calculating Note that F ₀ satisfies Ω ≧ | F ₀ |.

(vii) The sound source direction selection unit 18 searches for the speaker position r and the utterance direction θ at which the power aY (n, r, θ) averaged by frequency for each frame is maximum, and the power aY (n , r, θ) is determined as the estimated speech direction θ _out (n).

Hiroshi Nakajima, “Expanded Beamforming for Estimating Sound Source Direction”, Proceedings of the Acoustical Society of Japan, September 2005, p.619-620 Hiroshi Nakajima, 8 others, "Sound source directivity estimation using extended beamforming", Proceedings of the Acoustical Society of Japan, September 2005, p.621-622

The following two points can be cited as problems of the prior art.
(i) In order to respond to utterances at an arbitrary position and to estimate the direction of utterance with high accuracy, a large number of microphones are required, and it is necessary to devise the microphone installation positions.

In the prior art, the more accurate the utterance direction is estimated as the output power | Y (ω, n, r, θ) | ² of the fixed beamformer designed for each utterer position and utterance direction is different. can do. However, assuming a sound source with strong directivity in front of the mouth, such as a sound wave radiated from the mouth of the speaker, it is necessary to collect sound so as to surround the speaker with a large number of microphones 11 as shown in FIG. Depending on the speaker position and direction, the output power of the fixed beamformer does not differ, and the estimation error of the speech direction increases (for example, in the experiment of Non-Patent Document 2, 64 microphones are used). Therefore, in order to reduce the error, a large number of microphones are required, the apparatus becomes large, and it is difficult to use it by attaching it to a portable apparatus such as a telephone or an audio conference terminal.

(ii) High reverberation direction estimation performance cannot be obtained in a reverberant environment in which the reverberation time (the time from the arrival of the direct wave to the attenuation of 60 dB from the collected sound of the direct wave) is 250 msec or more.

  In a reverberant environment with a reverberation time of 250 msec or longer, it is difficult to design the acoustic propagation characteristics vH (ω, r, θ) with high accuracy because many strong reflected waves are mixed. Therefore, ambiguity occurs in the output of the fixed beamformer, and the estimation accuracy deteriorates. For example, in an actual environment room that is not subjected to low reverberation processing, reverberation time is generally about 250 to 500 msec, so that accurate estimation is difficult.

  An object of the present invention is to estimate the speech direction, which does not require a large number of microphones to be placed so as to surround the speaker, and can appropriately estimate the speech direction even in a reverberant environment with a reverberation time of 250 msec or more. To provide an apparatus, a method, and a program.

  The utterance direction estimation apparatus of the present invention includes an AD conversion unit, a frequency domain conversion unit, a correlation matrix calculation unit, an eigenvalue decomposition unit, an eigenvalue averaging processing unit, a front / horizontal cost calculation unit, and an utterance direction determination unit.

  The AD conversion unit converts an analog voice signal collected by a microphone array including M microphones (M is an integer of 2 or more) and a digital voice signal.

  The frequency domain transform unit transforms each digital audio signal from the time domain to the frequency domain.

  The correlation matrix calculation unit generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain.

  The eigenvalue decomposition unit decomposes the correlation matrix into an eigenvalue matrix that is a diagonal matrix with the squares of M eigenvalues as diagonal elements and an eigenvector matrix that includes M eigenvectors corresponding to the eigenvalues, Each of the eigenvalues is normalized and M normalized eigenvalues are output.

  The eigenvalue averaging processing unit outputs an averaged eigenvalue by taking a frequency average of the normalized eigenvalue obtained for each frequency.

The front / lateral cost calculation unit includes the average eigenvalue and a model average eigenvalue prepared in advance for each of a plurality of utterance directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r. From this, the front / side orientation determination cost is calculated and output for each utterance direction θ _j .

The utterance direction determination unit determines whether θ _j having the lowest front / horizontal direction determination cost corresponds to the front direction or the horizontal direction with respect to the microphone array, and outputs a determination result.

  According to the speech direction estimating apparatus of the present invention, it is not necessary to arrange a large number of microphones so as to surround a speaker, and it is possible to appropriately estimate the speech direction even in a reverberant environment where the reverberation time is 250 msec or more. It becomes.

The figure which shows the propagation characteristic of an audio | voice signal in a time domain. The image figure which represented typically the acoustic propagation vector group and eigenspace which comprise the correlation matrix showing the correlation between the signals picked up by each microphone for every three speech directions of front, left, and right. The image figure which shows the relationship between an utterance direction and an eigenvalue. The figure which shows the function structural example of the speech direction estimation apparatus of 1st Embodiment. The figure which shows the example of a processing flow of the speech direction estimation apparatus of 1st Embodiment. The image figure which shows the positional relationship of a microphone, a speaker, and a speech direction. The figure which shows the structural example which calculates | requires a model average eigenvalue. The figure which shows the function structural example of the speech direction estimation apparatus of 2nd Embodiment. The figure which shows the example of a processing flow of the utterance direction estimation apparatus of 2nd Embodiment. The figure which shows the structural example which calculates | requires a model average 1st eigenvector. The figure which shows the service structural example which incorporated this invention in the audio conference terminal. The figure which shows the function structural example of the speech direction estimation apparatus by a prior art. The image figure which shows the positional relationship of the microphone and speaker by a prior art.

[First Embodiment]
<Principle>
In the first embodiment, a configuration that makes it possible to estimate whether the utterance direction is the front direction or the horizontal direction with respect to the microphone array will be clarified. First, the principle of estimating the utterance direction in the front and side directions will be described.

  FIG. 1 shows the propagation characteristics of an audio signal in the time domain. Propagation characteristics can be broadly divided into three types: direct wave, initial reflected wave, and rear reverberation. In the time zone in which direct wave and initial reflected wave are observed, a microphone array composed of multiple microphones is used. It is known that waves with directionality are mixed. In particular, in the initial reverberation time zone (after the arrival of the direct wave, the time from the direct wave pickup power to the attenuation of 10 dB), a strong reflected wave with directionality is mixed, but the power of this reflected wave depends on the direction of speech. Change. Specifically, since the direct wave power increases as the utterance direction is the front direction, the reflected wave power decreases, and the direct wave power decreases as it is in the horizontal direction. The power of will increase. The present invention uses such a property to estimate the speech direction.

FIG. 2 shows acoustic propagation vector groups and eigenspaces (eigenvectors vV _i and eigenvalues λ _i) constituting a correlation matrix representing the correlation between signals picked up by each microphone for each of the three front, left, and right speech directions. (I-dimensional space formed by the above) is schematically represented. FIG. 2 shows a case where a microphone array is constituted by three microphones. Therefore, the acoustic propagation vector group and the eigenspace are expressed in three dimensions. In the case of facing front in FIG. 2, since many direct waves reach the microphone array and the arrival rate of the reflected waves is relatively low, the acoustic propagation vectors representing the direct waves are in the acoustic propagation vector group representing the reflected waves. Compared with greater power. At this time, the first eigenvalue λ ₁ of the correlation matrix is significantly larger than the second eigenvalue λ ₂ and the third eigenvalue λ ₃ . On the other hand, in the case of the horizontal orientation, the direct waves that reach the microphone array are reduced, and more reflected waves reach accordingly. Therefore, the power of the acoustic propagation vector that expresses the direct wave decreases, and the power of the acoustic propagation vector group that expresses the reflected wave increases. The first eigenvalue lambda ₁ when this is smaller than the front direction, the second eigenvalue lambda _{2 Conversely,} the third eigenvalue lambda ₃ is larger than that of the front facing. FIG. 3 shows an image of the difference that occurs in each eigenvalue between the case of facing front and the case of facing sideways. As described above, since the degree of arrival of the direct wave appears prominently in the eigenvalue λ _i (especially the first eigenvalue λ ₁ ) of the correlation matrix, by evaluating the value taken by the eigenvalue λ _i, Can be carved.

<Configuration>
FIG. 4 shows a functional configuration example of the speech direction estimating apparatus 100 of the present invention, and FIG. The utterance direction estimation device 100 estimates whether the utterance direction is a front direction or a horizontal direction with respect to the microphone array.

  The utterance direction estimation apparatus 100 includes a microphone array 101 including M microphones (M is an integer of 2 or more) 101-1 to 101-M, an AD conversion unit 12, a frequency domain conversion unit 13, and a correlation matrix calculation unit. 102, an eigenvalue decomposition unit 103, an eigenvalue averaging processing unit 104, a front / sideways cost calculation unit 105, and an utterance direction determination unit 106. Among them, the AD conversion unit 12 and the frequency conversion unit 13 are the same as those used in the speech direction estimation apparatus 10 described in the background art.

  In the prior art, as shown in FIG. 13, it has been necessary to arrange a large number of microphones so as to surround the speaker, but in the present invention, M microphones 101-1 to 101 -M are possible to the extent possible. It is sufficient to arrange them closely. Although the number of microphones constituting such a microphone array 101 is not too large, according to the configuration of the present invention described below, it is possible to estimate the utterance direction with two or more microphones. The arrangement may be two-dimensional or three-dimensional. By adopting a configuration in which a small number of microphones are arranged densely in this way, it can be attached to a portable device such as a telephone or an audio conference terminal, and the direction of speech by surrounding speakers can be estimated It becomes. The speaker speaks at a certain position r around the microphone array 101. FIG. 6 shows an image of a speaker talking around the microphone array 101 composed of seven microphones as viewed from above, and the direction of the arrow is the direction of speech. FIG. 6A shows a state where a speaker speaks in a front direction at each position, and FIG. 6B shows a state where a speaker speaks in a horizontal direction.

The AD converter 12 converts the analog voice signals uttered at the position r by the speaker 1 picked up by the M microphones 101-1 to 101 -M into digital voice signals X ₁ (t),. Convert to _M (t) (S1). Here, t represents an index of discrete time.

The frequency domain transform unit 13 receives as input a set (frame) of the above digital speech signals composed of a plurality of discrete time samples, and performs frequency domain digital speech signals X ₁ (ω, n),. It is converted to X _M (ω, n) and output (S2). Here, n represents a frame index, and ω represents a frequency index. The total number of frequency indexes is Ω.

The correlation matrix calculation unit 102 receives the digital audio signals X ₁ (ω, n),..., X _M (ω, n) in the frequency domain as inputs, and an M × M correlation matrix R representing the correlation between the signals. (ω, k) is sequentially generated and output for each frequency ω by equation (3) (S3).

R (ω, k) = E [vX (ω, n) · vX ^H (ω, n)] (3)
Here, vX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T
In Equation (3), H represents a conjugate transpose, and E calculated vX (ω, n) · vX (ω, n) ^H for each frame, and then calculated an expected value for each L frame by averaging processing or the like. Is an operator. That is, the correlation matrix is sequentially output at a rate of once per L frame, and k represents an output index of the correlation matrix. Note that L is preferably an integer greater than or equal to M.

The eigenvalue decomposition unit 103 receives the correlation matrix R (ω, k), and first, M eigenvalues λ ₁ (ω, k),..., Λ _M (ω, k) so as to satisfy Equation (4). ) Eigenvalue matrix Λ (ω, k), which is a diagonal matrix with each square as a diagonal element, and M eigenvectors vV ₁ (ω, k),..., VV _M (ω, k) Is decomposed into eigenvector matrix V (ω, k) by the eigenvalue decomposition method.

R (ω, k) = V (ω, k) ・ Λ (ω, k) ・ V ^H (ω, k) (4)
Here, Λ (ω, k) = diag [λ ₁ ² (ω, k),..., Λ _M ² (ω, k)]
λ ₁ (ω, k) ≧ λ ₂ (ω, k) ≧ ・ ・ ・ ≧ λ _M (ω, k)
V (ω, k) = [vV ₁ (ω, k),..., VV _M (ω, k)] ^T
vV _i (ω, k) = [V _{i, 1} (ω, k),..., V _{i, M} (ω, k)]
Then, each eigenvalue λ _i (ω, k) (i = 1, 2,..., M) is normalized by Expression (5), and a normalized eigenvalue nλ _i (ω, k) is output (S4). ).

When calculating the front / side determination cost based only on the first eigenvalue λ ₁ (ω, k) that is the maximum eigenvalue, only the normalized first eigenvalue nλ ₁ (ω, k) is calculated and output. It is good as well.

The eigenvalue averaging processing unit 104 averages the frequencies of the normalized eigenvalues nλ _i (ω, k) obtained for each frequency ω by the equation (6), and outputs the averaged eigenvalues aλ _i (k) (S5). ).

Here, F ₁ is the frequency index used for averaging, | F ₁ | is the total number of frequency indexes, and F ₁ is appropriately set so as to satisfy Ω ≧ | F ₁ |. When the front / side orientation determination cost is calculated based only on the first eigenvalue λ ₁ (ω, k), which is the maximum eigenvalue, only the averaged first eigenvalue aλ ₁ (k) may be output.

The front / horizontal cost calculation unit 105 averaged eigenvalue sequence vaλ (k) = [aλ ₁ (k), aλ ₂ (k),..., Aλ _M (k) obtained by the eigenvalue averaging processing unit 104. ] And a model-averaged eigenvalue sequence vaQ (k, r, θ _j ) = prepared in advance for each of a plurality of utterance directions θ _j (j = 1, 2,..., N, N ≧ 2) at the utterance position r = From [aQ ₁ (k, r, θ _j ), aQ ₂ (k, r, θ _j ),..., AQ _M (k, r, θ _j )], the front and side directions for each utterance direction θ _j The judgment cost C ₁ (k, r, θ _j ) is calculated and output (S6). Here, the model averaged eigenvalue aQ _i (k, r, θ _j ) is converted into a speech signal uttered for each direction θ _j at the utterance position r under the same configuration as in FIG. On the other hand, it can be obtained by performing processing up to the eigenvalue averaging processing unit 104. The model averaged eigenvalue aQ _i (k, r, θ _j ) may be input to the front / sideways cost calculation unit 105 by an arbitrary method such as recording in a database in advance and reading from the database.
The front / side orientation determination cost C ₁ (k, r, θ _j ) is obtained by equation (7).

Note that the difference in eigenvalue between the case of speaking in front and the case of speaking in side is reflected particularly prominently in the first eigenvalue, so the front / side determination cost C ₁ (k, r, θ _j ) is expressed as follows. You may obtain | require by Formula (8) only from a 1st eigenvalue.

The front / side determination cost C ₁ (k, r, θ _j ) is an index representing the proximity between the utterance direction to be determined and each utterance direction θ _j prepared in advance. This means that the direction of the target utterance is close to θ _j . That is, it is possible to estimate the utterance direction as a determination target by extracting θ _j having the lowest cost from each θ _j prepared in advance.

Utterance orientation determining unit 106 is a front-lateral determining cost _{C 1 (k, r, θ} j) is the smallest value becomes theta _j is whether the corresponding sideways or corresponds to the front direction to the microphone array 101 The determination is made and the determination result is output (S7). For example, with respect to the microphone array 101 from the speech position r, the front direction is set to 0 °, and the horizontal direction is set to other angles, and three models of θ ₁ = 0 °, θ ₂ = −90 °, and θ ₃ = + 90 ° are modeled. When the average eigenvalue sequence is prepared, when the front / side determination cost is C ₁ (k, r, θ ₁ ) <C ₁ (k, r, θ ₂ ) <C ₁ (k, r, θ ₃ ) Alternatively, when C ₁ (k, r, θ ₁ ) <C ₁ (k, r, θ ₃ ) <C ₁ (k, r, θ ₂ ), it is determined to face front, and otherwise, it is determined to be sideways.

  As described above, the utterance direction estimation apparatus according to the first embodiment can estimate whether the utterer speaks in the front direction or the side direction with respect to the microphone array. In addition, since the microphone array may be configured in a form in which a small number of microphones are densely packed, it is possible to configure the microphone array compactly without enclosing the speaker. In addition, since the reverberation is actively used, it is possible to appropriately estimate the utterance direction even in a reverberation environment where the reverberation time is 250 msec or more. In addition, since the eigenvalue decomposition process, which is the core of the process in the present invention, has a small amount of calculation, it is advantageous when it is incorporated in equipment with low CPU specifications such as a portable terminal.

[Second Embodiment]
In the first embodiment, when model averaged eigenvalue sequences are prepared for three orientations, for example, θ ₁ = 0 °, θ ₂ = negative angle, and θ ₃ = positive angle, the negative angle is leftward and the positive angle If it is considered to be rightward, it seems that it is possible to determine not only front and sideways but also left and right. However, in reality, C ₁ (k, r, θ ₂ ) ≈C ₁ (k, r, θ ₃ ) is obtained at angles other than the vicinity of the front, and it is difficult to correctly determine the left-right direction. Therefore, in the second embodiment, it is possible to further determine whether the horizontal orientation is leftward or rightward.

<Principle>
As mentioned in the explanation of the principle of the first embodiment, strong reflected waves having directivity with respect to the microphone array are mixed in the initial reverberation time zone, and the power of the reflected waves changes depending on the direction of speech.

When comparing the left direction and the right direction in FIG. 2, there is almost no difference in the acoustic propagation vectors constituting the direct wave and the rear reverberation, but the acoustic propagation vectors constituting the initial reflected wave are different. This is because the directionality of the strong reflected wave from the wall mixed in the observation signal differs depending on the direction of speech. That is, the direction and power of acoustic propagation constituting the initial reflected wave change depending on whether the utterance direction is left or right, and how the eigenspace is stretched also changes. Since the influence of this change is prominent in the eigenvector vV _{i of} the correlation matrix, particularly the first eigenvector vV ₁ corresponding to the largest eigenvalue of the correlation matrix, the value taken by the first eigenvector vV ₁ is evaluated. , It is possible to distinguish whether the utterance direction is leftward or rightward.

<Configuration>
FIG. 8 shows a functional configuration example of the speech direction estimating apparatus 200 of the present invention, and FIG. 9 shows a processing flow example thereof.

  The utterance direction estimation apparatus 200 includes a microphone array 101 including M (M is an integer of 2 or more) microphones 101-1 to 101-M, an AD converter 12, a frequency domain converter 13, and a correlation matrix calculator. 102, an eigenvalue decomposition unit 201, an eigenvalue averaging processing unit 104, a front / horizontal cost calculation unit 105, a first eigenvector averaging processing unit 202, a left / right direction cost calculation unit 203, and an utterance direction determination unit 204 Is provided. Among them, the same names and symbols as described in the first embodiment are attached except for the eigenvalue decomposition unit 201, the first eigenvector averaging processing unit 202, the left-right direction cost calculation unit 203, and the speech direction determination unit 204. Since it is the same as a component, description of a function / process is abbreviate | omitted.

The eigenvalue decomposition unit 201 performs the same decomposition process as the eigenvalue decomposition unit 103 of the first embodiment, outputs a normalized eigenvalue nλ _i (ω, k), and the first eigenvalue λ that is the maximum eigenvalue. _{1 (ω,} k) first eigenvector vV _{1 (ω,} k) corresponding to output (S11).

The first eigenvector averaging processing unit 202 obtains and outputs an averaged first eigenvector vaV ₁ (k) by taking a frequency average for the first eigenvector vV ₁ (ω, k) obtained for each frequency ω. (S12). The first eigenvector vV 1 _(ω, k) is a vector of complex domain, it is not possible to perform frequency averaging process simple product sum operation for frequency dependent omega. Therefore, the first eigenvector vV ₁ (ω, k) is converted into a frequency-independent feature amount with reference to the frequency normalization method disclosed in Japanese Patent Application Laid-Open No. 2007-226036 (paragraphs [0078] [0079] etc.). The frequency averaging process is performed above.

Specifically, first, the first eigenvector vV ₁ (ω, k) = [V _1,1 (ω, k),..., V _{1, M} (ω, k)] is expressed by equations (9) (10 ), The feature vector vP ₁ (ω, k) = [P _1,1 (ω, k),..., P _{1, M} (ω, k)] that measures the similarity of the frequency-independent acoustic propagation characteristics. Convert to

Here, i = 1, 2,..., M, ξ _i (ω, k) is a complex rotator, arg [•] is an operator for calculating a phase angle, and f _ω corresponds to a frequency index ω. Frequency (Hz), d is the maximum distance (m) of the microphone array, and c is the speed of sound (m / s).

Then, the obtained feature vector vP ₁ (ω, k) = [P _1,1 (ω, k),..., P _{1, M} (ω, k)] is frequency-averaged by the equation (11). Processing is performed to output an averaged first eigenvector vaV ₁ (k) = [aV _1,1 (k),..., Av _{1, M} (k)].

Here, F ₁ is a frequency index used in frequency averaging, | F ₁ | is the total number of frequency indexes used in frequency averaging processing, and F ₁ is appropriately set so as to satisfy Ω ≧ | F ₁ |.

The left-right cost calculation unit 203 calculates the averaged first eigenvector vaV ₁ (k) = [aV _1,1 (k),..., AV _{1, M} (k) obtained by the first eigenvector averaging processing unit 202. )] And a model-averaged first eigenvector vaS ₁ (k, r, θ) prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the speech position r. _j ) = [aS _1,1 (k, r, θ _j ),..., aS _{1, M} (k, r, θ _j )]], the left-right determination cost C ₂ (for each utterance direction θ _j k, r, θ _j ) is calculated and output (S13). Here, the utterance orientation theta model averaging first eigenvector for each _{j vaS 1 (k, r,} θ j) , the orientation theta _j in speech position r under the same configuration as FIG. 8 as shown in FIG. 10 for example It can be obtained by performing the processing up to the first eigenvector averaging processing unit 202 for each voice signal uttered. The model averaged first eigenvector vaS ₁ (k, r, θ _j ) may be input to the left-right cost calculation unit 203 by an arbitrary method such as recording in a database in advance and reading from the database.
The left / right direction determination cost C ₂ (k, r, θ _j ) is obtained by Expression (12).

Utterance orientation determining unit 204, a front-transverse determined cost C ₁ for each _{θ j (k, r, θ} j) is the sum of the left and right orientation determining cost _{C 2 (k, r, θ} j) C (k , r, θ _j ), the minimum of all combinations of the sums of the front / side direction determination costs C ₁ (k, r, θ _j ) and the left / right direction determination costs C ₂ (k, r, θ _j ) It is determined whether θ _j that is the utterance direction of C (k, r, θ _j ) closest to the value corresponds to the front, left, or right direction with respect to the microphone array, and a determination result is output. (S14).

For example, the model eigenvalue and the model first eigenvector are expressed as follows: θ ₁ = 0 ° (front direction), θ ₂ = −90 ° (left direction), θ ₃ = + 90 ° (right direction) with respect to the microphone array 101 from the speech position r. Consider the case where three orientations are prepared. In this case, three costs of C ₁ (k, r, θ ₁ ), C ₁ (k, r, θ ₂ ), and C ₁ (k, r, θ ₃ ) are output from the front / lateral cost calculation unit 105. Then, the three costs of C ₂ (k, r, θ ₁ ), C ₂ (k, r, θ ₂ ), and C ₂ (k, r, θ ₃ ) are also output from the left-right cost calculation unit 203. . The speech direction determination unit 204 receives these as inputs, and C (k, r, θ _j ) = C ₁ (k, r, θ _j ) + C ₂ (k, r, θ _j ) ₁ ), C (k, r, θ ₂ ), and C (k, r, θ ₃ ), respectively. The three cost C obtained (k, r, θ _j) of the minimum cost min {C (k, r, θ j)} a theta _j of estimating the orientation of the utterance to be determined. In this example, if C (k, r, θ ₁ ) is the minimum cost, it faces forward, if C (k, r, θ ₂ ) has the minimum cost, it faces left, C (k, r, θ ₃ ) Can be estimated to be right-facing.

  As described above, according to the speech direction estimation apparatus of the second embodiment, in addition to the effect of the configuration of the first embodiment, it is possible to determine whether the horizontal orientation is further left or right. Communication can be performed more smoothly.

  When the configuration of the utterance direction estimation device of each of the above embodiments is realized by a computer, the processing contents of the functions that each device should have are described by a program. The processing functions are realized on the computer by executing the program on the computer. In this case, at least a part of the processing content may be realized by hardware.

  Further, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

[Service application example]
FIG. 11 shows a configuration example of a service in which the present invention is incorporated into an audio conference terminal. Assume that the conference hall A and the conference hall B are connected by a voice terminal through a network. Speech direction information is extracted from a voice signal picked up by a microphone attached to the voice conference terminal, and transmitted to the other party along with the voice information. By presenting the utterance direction information as visual information on the other party side, it is possible to convey the situation of the place that is difficult to convey only with the voice information.

  In addition, the utterance direction estimation technology can be applied to a minutes system that records the state of communication in a meeting using video and audio. In other words, the utterance direction estimation technique makes it possible to tag the recorded voice and video as to who spoke to whom, which is useful for organizing the minutes.

  In addition, images for services that detect faces from images, for example, surveillance used for surveillance cameras and intercoms, security purposes, and services that determine whether you are paying attention to advertising with digital signage, etc. It is possible to replace the detection of the direction by the detection with an audio signal.

Claims (9)

An analog-to-digital conversion unit that converts analog audio signals collected by a microphone array composed of M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain converter for converting each of the digital audio signals from the time domain to the frequency domain;
A correlation matrix calculator that generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix, which is a diagonal matrix having the squares of M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues. An eigenvalue decomposition unit that normalizes and outputs M normalized eigenvalues;
About the normalized eigenvalue obtained for each frequency, an eigenvalue averaging processing unit that outputs an averaged eigenvalue by taking a frequency average;
From the average eigenvalue and the model average eigenvalue prepared in advance for each of a plurality of utterance directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, the front / side determination cost and the front-lateral cost calculator C 1 a _{(theta j)} and outputs the calculated for each of the speech direction theta _j,
An utterance direction determination unit that determines whether θ _j having the smallest front / horizontal determination cost C ₁ (θ _j ) corresponds to the front direction or the horizontal direction with respect to the microphone array and outputs a determination result; ,
An utterance direction estimation device comprising: An analog-to-digital conversion unit that converts analog audio signals collected by a microphone array composed of M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain converter for converting each of the digital audio signals from the time domain to the frequency domain;
A correlation matrix calculator that generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix, which is a diagonal matrix having the squares of M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues. And an eigenvalue decomposition unit for outputting an eigenvector corresponding to the largest eigenvalue among the M eigenvalues (hereinafter referred to as “first eigenvector”);
About the normalized eigenvalue obtained for each frequency, an eigenvalue averaging processing unit that outputs an averaged eigenvalue by taking a frequency average;
From the average eigenvalue and the model average eigenvalue prepared in advance for each of a plurality of utterance directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, the front / side determination cost A front and side cost calculator that calculates and outputs C ₁ (θ _j ) for each utterance direction θ _j ;
A first eigenvector averaging processing unit that outputs an averaged first eigenvector by taking a frequency average for the first eigenvector obtained for each frequency;
From the averaged first eigenvector and the model averaged first eigenvector prepared in advance for each of the plurality of utterance directions θ _{j at the} position r, the left-right direction determination cost C ₂ (θ _j ) is calculated for each utterance direction θ _j. A left-right cost calculator that calculates and outputs
Sum of each theta _j the front-lateral determining cost _{C 1} (θ _j) and the left-right orientation determining cost _{C 2 (θ j) C (} θ j) = C 1 (θ j) + C 2 (θ j) Utterance direction in which θ _j , which is the smallest C (θ _j ) utterance direction, corresponds to the front, left, or right direction with respect to the microphone array and outputs a determination result A determination unit;
An utterance direction estimation device comprising: In the utterance direction estimation device according to claim 2,
The averaged first eigenvector is obtained by calculating a feature value representing similarity of acoustic propagation characteristics independent of frequency for each of the M elements constituting the first eigenvector, and then taking a frequency average for the feature value. The utterance direction estimation device obtained by In the utterance direction estimation apparatus according to any one of claims 1 to 3,
The utterance direction estimation device that calculates the front / side orientation determination cost C ₁ (θ _j ) from the averaged first eigenvalue and the model averaged first eigenvalue corresponding to the maximum of the M eigenvalues. An AD conversion step for converting analog audio signals collected by a microphone array comprising M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain transforming step for transforming each digital audio signal from the time domain to the frequency domain;
A correlation matrix calculation step of generating and outputting an M × M correlation matrix representing a correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix, which is a diagonal matrix having the squares of M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues, and the eigenvalues are normalized. An eigenvalue decomposition step that outputs M normalized eigenvalues,
An eigenvalue averaging process step of outputting an averaged eigenvalue by taking a frequency average for the normalized eigenvalue obtained for each frequency;
From the average eigenvalue and the model average eigenvalue prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, the front / side orientation determination cost C A front and side cost calculation step of calculating and outputting ₁ (θ _j ) for each utterance direction θ _j ,
An utterance direction determination step of determining whether θ _j having the smallest front / horizontal determination cost C ₁ (θ _j ) corresponds to the front direction or the horizontal direction with respect to the microphone array, and outputting a determination result;
Utterance direction estimation method. An AD conversion step for converting analog audio signals collected by a microphone array comprising M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain transforming step for transforming each digital audio signal from the time domain to the frequency domain;
A correlation matrix calculation step of generating and outputting an M × M correlation matrix representing a correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix, which is a diagonal matrix having the squares of M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues, and the eigenvalues are normalized. An eigenvalue decomposition step of outputting M normalized eigenvalues and outputting an eigenvector corresponding to the largest eigenvalue among the M eigenvalues (hereinafter referred to as “first eigenvector”);
An eigenvalue averaging process step of outputting an averaged eigenvalue by taking a frequency average for the normalized eigenvalue obtained for each frequency;
From the average eigenvalue and the model average eigenvalue prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, the front / side orientation determination cost C A front and side cost calculation step of calculating and outputting ₁ (θ _j ) for each utterance direction θ _j ,
A first eigenvector averaging processing step of outputting an averaged first eigenvector by taking a frequency average of the first eigenvector obtained for each frequency;
From the averaged first eigenvector and the model averaged first eigenvector prepared in advance for each of the plurality of utterance directions θ _{j at the} position r, the left / right direction determination cost C ₂ (θ _j ) is determined for each utterance direction θ _j . Left and right cost calculation step to calculate and output,
Sum of each theta _j the front-lateral determining cost _{C 1} (θ _j) and the left-right orientation determining cost _{C 2 (θ j) C (} θ j) = C 1 (θ j) + C 2 (θ j) Utterance direction in which θ _j , which is the smallest C (θ _j ) utterance direction, corresponds to the front, left, or right direction with respect to the microphone array and outputs a determination result A determination step;
Utterance direction estimation method. In the speech direction estimation method according to claim 6,
The averaged first eigenvector is obtained by calculating a feature value representing similarity of acoustic propagation characteristics independent of frequency for each of the M elements constituting the first eigenvector, and then taking a frequency average for the feature value. The utterance direction estimation method obtained by In the speech direction estimation method according to any one of claims 5 to 7,
The speech direction estimation method in which the front / side orientation determination cost C ₁ (θ _j ) is calculated from the averaged first eigenvalue and the model averaged first eigenvalue corresponding to the maximum of the M eigenvalues.   A program for causing a computer to function as the apparatus according to claim 1.

Images