JP2010206392A - Speech direction estimation device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech direction estimation device which does not request to arrange many microphones so as to enclose a speaker, and can appropriately estimate the speech direction even under environment in which a reverberation time is long. <P>SOLUTION: The plurality of microphones pick up a sound signal originated by the speaker around a microphone array which is composed of the plurality of microphones. A correlation matrix which represents correlation between voice signals each picked up by each microphone, is created, and it is estimated that the speaker has spoken from what direction to the microphone array from an eigenvector obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for estimating the utterance direction of a speaker from an audio signal input to a microphone.

  A system for exchanging voice information such as a telephone or a voice conference terminal is generally called a voice communication system. In the video conference system, the video is added to the audio information and presented, so that the situation of the place is easily transmitted, but in the audio communication system, it is difficult to grasp the situation of the other party. One of the information on the other party's situation is utterance direction information. By receiving this information from the other party, it is possible to grasp the direction in which the speaker is speaking and to facilitate communication.

  Conventional techniques for estimating such speech direction information are disclosed in Non-Patent Documents 1 and 2 and the like, and a configuration example is shown in FIG. The speech direction estimation apparatus 10 in this configuration example estimates speech direction information as follows.

(i) The voice from the speaker 1 is picked up using M (M is an integer of 2 or more) microphones 11-1,. The collected analog signal is converted by the AD converter 12 into a digital signal vX (t) = [X ₁ (t),..., X _M (t)] ^T. Here, t represents an index of discrete time.

(ii) The frequency domain transform unit 13 receives a set (frame) of the digital signals composed of a plurality of samples as an input, and performs frequency domain signal vX (ω, n) = [X ₁ (ω, n) by fast Fourier transform or the like. _{, ···, X M (ω,} n)] to convert to ^T. Here, ω represents a frequency index, and the total number of frequency indexes is Ω. N represents the index of the frame.

(iii) In the fixed beamformer design unit 14, the fixed beamformer vG (ω, r, θ) = [G ₁ (ω, r, θ),..., G _M ( ω, r, θ)] ^T is designed. G _i (ω, r, θ) is a coefficient that is multiplied by the frequency component X _i (ω, n) of the i-th microphone in order to emphasize / suppress the sound source at the speaker position r and the speech direction θ.

In designing, acoustic propagation characteristics between the sound source and the microphone vH (ω, r, θ) = [H ₁ (ω, r, θ),..., H _{M for} each predetermined speaker position and direction. (ω, r, θ)] ^T is obtained using a simulation value or an actual measurement value. Here, H _i (ω, r, θ) represents acoustic propagation characteristics between the sound source at the speaker position r and the speech direction θ and the i-th microphone.

  The fixed beamformer vG (ω, r, θ) is designed as a value satisfying the expressions (1) and (2) representing the relationship with the acoustic propagation characteristics.

vH (ω, r _T , θ _T ) ^H · vG (ω, r _T , θ _T ) = 1 (1)
vH (ω, r _U , θ _U ) ^H · vG (ω, r _T , θ _T ) = 0 (2)
Expressions (1) and (2) emphasize the output power of the speaker position r _T and the speech direction θ _T , and the fixed beamformer so as to suppress the output power of the other speaker positions r _U and the speech direction θ _U. It shows that vG (ω, r, θ) is designed.

(iv) In the product-sum calculation unit 15, the frequency domain signal vX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T and the fixed beamformer vG (ω , r, θ) = [G 1 (ω, r, θ), ···, G M (ω, r, θ)] as input ^T, each frequency ω, speaker position r, every utterance direction θ Multiplying the frequency component X _i (ω, n) corresponding to each microphone by the coefficient G _i (ω, r, θ) of the fixed beamformer, and adding the obtained M components, the output Y (ω, n, r, θ) is calculated. This calculation is synonymous with calculating Y (ω, n, r, θ) = vG (ω, r, θ) ^H · vX (ω, n).

(v) The power calculator 16 calculates and outputs power | Y (ω, n, r, θ) | ² from the output Y (ω, n, r, θ) from the product-sum calculator 15.

(vi) The frequency averaging processing unit 17 averages the power | Y (ω, n, r, θ) | ² output from the power calculation unit 16 by frequency, and aY (n, r, θ) is obtained. obtain. In this calculation, if F ₀ is defined as the frequency index used in the averaging process, and | F ₀ | is defined as the total number of frequency indexes,

を計算することと同義である。なお、Ｆ_０はΩ≧|Ｆ_０|を満たす。

Is equivalent to calculating Note that F ₀ satisfies Ω ≧ | F ₀ |.

(vii) The sound source direction selection unit 18 searches for the speaker position r and the utterance direction θ at which the power aY (n, r, θ) averaged by frequency for each frame is maximum, and the power aY (n , r, θ) is determined as the estimated speech direction θ _out (n).

Hiroshi Nakajima, “Expanded Beamforming for Estimating Sound Source Direction”, Proceedings of the Acoustical Society of Japan, September 2005, p.619-620 Hiroshi Nakajima, 8 others, "Sound source directivity estimation using extended beamforming", Proceedings of the Acoustical Society of Japan, September 2005, p.621-622

The following two points can be cited as problems of the prior art.
(i) In order to respond to utterances at an arbitrary position and to estimate the direction of utterance with high accuracy, a large number of microphones are required, and it is necessary to devise the microphone installation positions.

In the prior art, the more accurate the utterance direction is estimated as the output power | Y (ω, n, r, θ) | ² of the fixed beamformer designed for each utterer position and utterance direction is different. can do. However, assuming a sound source with strong directivity in front of the mouth, such as a sound wave radiated from the mouth of the speaker, it is necessary to collect sound so as to surround the speaker with a large number of microphones 11 as shown in FIG. Depending on the speaker position and direction, the output power of the fixed beamformer does not differ, and the estimation error of the speech direction increases (for example, in the experiment of Non-Patent Document 2, 64 microphones are used). Therefore, in order to reduce the error, a large number of microphones are required, the apparatus becomes large, and it is difficult to use it by attaching it to a portable apparatus such as a telephone or an audio conference terminal.

(ii) High reverberation direction estimation performance cannot be obtained in a reverberant environment where the reverberation time (the time from the direct wave arrival time to the 60 dB attenuation from the direct wave pickup power) is 250 msec or more. Below, since many strong reflected waves are mixed, it is difficult to design the acoustic propagation characteristic vH (ω, r, θ) with high accuracy. Therefore, ambiguity occurs in the output of the fixed beamformer, and the estimation accuracy deteriorates. For example, in an actual environment room that is not subjected to low reverberation processing, reverberation time is generally about 250 to 500 msec, so that accurate estimation is difficult.

  An object of the present invention is to estimate the speech direction, which does not require a large number of microphones to be placed so as to surround the speaker, and can appropriately estimate the speech direction even in a reverberant environment with a reverberation time of 250 msec or more. To provide an apparatus, a method, and a program.

  The speech direction estimation apparatus of the present invention includes an AD conversion unit, a frequency domain conversion unit, a correlation matrix calculation unit, an eigenvalue decomposition unit, a first eigenvector averaging processing unit, a left-right cost calculation unit, and a speech direction determination unit. .

  The AD conversion unit converts an analog voice signal collected by a microphone array including M microphones (M is an integer of 2 or more) and a digital voice signal.

  The frequency domain transform unit transforms each digital audio signal from the time domain to the frequency domain.

  The correlation matrix calculation unit generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain.

  The eigenvalue decomposition unit decomposes the correlation matrix into an eigenvalue matrix that is a diagonal matrix with the squares of M eigenvalues as diagonal elements and an eigenvector matrix that includes M eigenvectors corresponding to the eigenvalues, An eigenvector corresponding to the largest eigenvalue (hereinafter referred to as “first eigenvector”) is output.

  The first eigenvector averaging processing unit outputs the averaged first eigenvector by taking the frequency average of the first eigenvector obtained for each frequency.

The left-right direction cost calculation unit calculates the model-averaged first prepared for each of the averaged first eigenvectors and a plurality of utterance directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r. From one eigenvector, the right / left direction determination cost is calculated for each utterance direction θ _j and output.

The utterance direction determination unit determines whether θ _j having the smallest left-right direction determination cost corresponds to the left direction or the right direction with respect to the microphone array, and outputs a determination result.

  According to the speech direction estimating apparatus of the present invention, it is not necessary to arrange a large number of microphones so as to surround a speaker, and it is possible to appropriately estimate the speech direction even in a reverberant environment where the reverberation time is 250 msec or more. It becomes.

The figure which shows the propagation characteristic of an audio | voice signal in a time domain. The image figure which represented typically the acoustic propagation vector group and eigenspace which comprise the correlation matrix showing the correlation between the signals picked up by each microphone for every three speech directions of front, left, and right. The figure which shows the function structural example of the speech direction estimation apparatus of 1st Embodiment. The figure which shows the example of a processing flow of the speech direction estimation apparatus of 1st Embodiment. The image figure which shows the positional relationship of a microphone, a speaker, and a speech direction. The figure which shows the structural example which calculates | requires a model average 1st eigenvector. The image figure which shows the relationship between an utterance direction and an eigenvalue. The figure which shows the function structural example of the speech direction estimation apparatus of 2nd Embodiment. The figure which shows the example of a processing flow of the utterance direction estimation apparatus of 2nd Embodiment. The figure which shows the structural example which calculates | requires a model average eigenvalue. The figure which shows the service structural example which incorporated this invention in the audio conference terminal. The figure which shows the function structural example of the speech direction estimation apparatus by a prior art. The image figure which shows the positional relationship of the microphone and speaker by a prior art.

[First Embodiment]
<Principle>
In the first embodiment, a configuration that makes it possible to estimate whether the utterance direction is leftward or rightward with respect to the microphone array will be clarified. First, the principle of estimating the left-right direction of speech will be described.

  FIG. 1 shows the propagation characteristics of an audio signal in the time domain. Propagation characteristics can be broadly divided into three types: direct wave, initial reflected wave, and rear reverberation. In the time zone in which direct wave and initial reflected wave are observed, a microphone array composed of multiple microphones is used. It is known that waves with directionality are mixed. In particular, in the initial reverberation time zone (after the arrival of the direct wave, the time from the direct wave pickup power to the attenuation of 10 dB), a strong reflected wave with directionality is mixed, but the power of this reflected wave depends on the direction of speech. Change.

FIG. 2 shows acoustic propagation vector groups and eigenspaces (eigenvectors vV _i and eigenvalues λ _i) constituting a correlation matrix representing the correlation between signals picked up by each microphone for each of the three front, left, and right speech directions. (I-dimensional space formed by the above) is schematically represented. FIG. 2 shows a case where a microphone array is constituted by three microphones. Therefore, the acoustic propagation vector group and the eigenspace are expressed in three dimensions. When comparing the left direction and the right direction in FIG. 2, there is almost no difference in the acoustic propagation vectors constituting the direct wave and the rear reverberation, but the acoustic propagation vectors constituting the initial reflected wave are different. This is because the directionality of the strong reflected wave from the wall mixed in the observation signal differs depending on the direction of speech. That is, the direction and power of acoustic propagation constituting the initial reflected wave change depending on whether the utterance direction is left or right, and how the eigenspace is stretched also changes. Since the influence of this change is prominent in the eigenvector vV _{i of} the correlation matrix, particularly the first eigenvector vV ₁ corresponding to the largest eigenvalue of the correlation matrix, the value taken by the first eigenvector vV ₁ is evaluated. , It is possible to distinguish whether the utterance direction is leftward or rightward.

<Configuration>
FIG. 3 shows a functional configuration example of the speech direction estimating apparatus 100 of the present invention, and FIG. 4 shows a processing flow example thereof. The utterance direction estimation apparatus 100 includes a microphone array 101 including M microphones (M is an integer of 2 or more) 101-1 to 101-M, an AD conversion unit 12, a frequency domain conversion unit 13, and a correlation matrix calculation unit. 102, an eigenvalue decomposition unit 103, a first eigenvector averaging processing unit 104, a left-right direction cost calculation unit 105, and an utterance direction determination unit 106. Among them, the AD conversion unit 12 and the frequency conversion unit 13 are the same as those used in the speech direction estimation apparatus 10 described in the background art.

  In the prior art, as shown in FIG. 13, it has been necessary to arrange a large number of microphones so as to surround the speaker, but in the present invention, M microphones 101-1 to 101 -M are possible to the extent possible. It is sufficient to arrange them closely. Although the number of microphones constituting such a microphone array 101 is not too large, according to the configuration of the present invention described below, it is possible to estimate the utterance direction with two or more microphones. The arrangement may be two-dimensional or three-dimensional. By adopting a configuration in which a small number of microphones are arranged densely in this way, it can be attached to a portable device such as a telephone or an audio conference terminal, and the direction of speech by surrounding speakers can be estimated It becomes. The speaker speaks at a certain position r around the microphone array 101. FIG. 5 shows an image of a speaker talking around a microphone array 101 composed of seven microphones as viewed from above, and the direction of the arrow is the direction of speech. FIG. 5A shows a state where the speaker is speaking leftward at each position, and FIG. 5B shows a state where the speaker is speaking rightward.

The AD converter 12 converts the analog voice signals uttered at the position r by the speaker 1 picked up by the M microphones 101-1 to 101 -M into digital voice signals X ₁ (t),. Convert to _M (t) (S1). Here, t represents an index of discrete time.

The frequency domain transform unit 13 receives as input a set (frame) of the above digital speech signals composed of a plurality of discrete time samples, and performs frequency domain digital speech signals X ₁ (ω, n),. It is converted to X _M (ω, n) and output (S2). Here, n represents a frame index, and ω represents a frequency index. The total number of frequency indexes is Ω.

The correlation matrix calculation unit 102 receives the digital audio signals X ₁ (ω, n),..., X _M (ω, n) in the frequency domain as inputs, and an M × M correlation matrix R representing the correlation between the signals. (ω, k) is sequentially generated and output for each frequency ω by equation (3) (S3).

R (ω, k) = E [vX (ω, n) · vX ^H (ω, n)] (3)
Here, vX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T
In Equation (3), H represents a conjugate transpose, and E calculated vX (ω, n) · vX (ω, n) ^H for each frame, and then calculated an expected value for each L frame by averaging processing or the like. Is an operator. That is, the correlation matrix is sequentially output at a rate of once per L frame, and k represents an output index of the correlation matrix. Note that L is preferably an integer greater than or equal to M.

The eigenvalue decomposition unit 103 receives the correlation matrix R (ω, k), and first, M eigenvalues λ ₁ (ω, k),..., Λ _M (ω, k) so as to satisfy Equation (4). ) Eigenvalue matrix Λ (ω, k), which is a diagonal matrix with each square as a diagonal element, and M eigenvectors vV ₁ (ω, k),..., VV _M (ω, k) Is decomposed into eigenvector matrix V (ω, k) by the eigenvalue decomposition method.

R (ω, k) = V (ω, k) ・ Λ (ω, k) ・ V ^H (ω, k) (4)
Here, Λ (ω, k) = diag [λ ₁ ² (ω, k),..., Λ _M ² (ω, k)]
λ ₁ (ω, k) ≧ λ ₂ (ω, k) ≧ ・ ・ ・ ≧ λ _M (ω, k)
V (ω, k) = [vV ₁ (ω, k),..., VV _M (ω, k)] ^T
vV _i (ω, k) = [V _{i, 1} (ω, k),..., V _{i, M} (ω, k)]
Note that diag [•] is an operator having the components in [•] as elements of a diagonal matrix. Then, the first eigenvector vV ₁ (ω, k) corresponding to the first eigenvalue λ ₁ (ω, k) which is the maximum eigenvalue is output (S4).

The first eigenvector averaging processing unit 104 obtains and outputs an averaged first eigenvector vaV ₁ (k) by taking a frequency average for the first eigenvector vV ₁ (ω, k) obtained for each frequency ω. (S5). The first eigenvector vV 1 _(ω, k) is a vector of complex domain, it is not possible to perform frequency averaging process simple product sum operation for frequency dependent omega. Therefore, the first eigenvector vV ₁ (ω, k) is converted into a frequency-independent feature amount with reference to the frequency normalization method disclosed in Japanese Patent Application Laid-Open No. 2007-226036 (paragraphs [0078] [0079] etc.). The frequency averaging process is performed above.

Specifically, first, the first eigenvector vV ₁ (ω, k) = [V _1,1 (ω, k),..., V _{1, M} (ω, k)] is expressed by equations (5) (6 ) by the feature vector vP 1 to measure the similarity of acoustic propagation characteristics not dependent on the frequency _{(ω, k) = [P} 1,1 (ω, k), ···, P 1, M (ω, k)] Convert to

Here, i = 1, 2,..., M, ξ _i (ω, k) is a complex rotator, arg [•] is an operator for calculating a phase angle, and f _ω corresponds to a frequency index ω. Frequency (Hz), d is the maximum distance (m) of the microphone array, and c is the speed of sound (m / s).

Then, the obtained feature vector vP ₁ (ω, k) = [P _1,1 (ω, k),..., P _{1, M} (ω, k)] is frequency-averaged using equation (7). Processing is performed, and the averaged first eigenvector vaV ₁ (k) = [aV _1,1 (k),..., AV _{1, M} (k)] is output.

Here, F ₁ is a frequency index used in frequency averaging, | F ₁ | is the total number of frequency indexes used in frequency averaging processing, and F ₁ is appropriately set so as to satisfy Ω ≧ | F ₁ |.

The left-right cost calculator 105 calculates the averaged first eigenvector vaV ₁ (k) = [aV _1,1 (k),..., AV _{1, M} (k) obtained by the first eigenvector averaging processor 104. )] And a model-averaged first eigenvector vaS ₁ (k, r, θ) prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the speech position r. _j ) = [aS _1,1 (k, r, θ _j ),..., aS _{1, M} (k, r, θ _j )]], the left-right determination cost C ₁ (for each utterance direction θ _j k, r, θ _j ) is calculated and output (S6). Here, the utterance orientation theta model averaging first eigenvector for each _{j vaS 1 (k, r,} θ j) , the orientation theta _j in speech position r under the same configuration as FIG. 3, for example, as shown in FIG. 6 It can be obtained by performing the processing up to the first eigenvector averaging processing unit 104 for each voice signal uttered. Note that the input of the model averaged first eigenvector vaS ₁ (k, r, θ _j ) to the left-right cost calculation unit 105 may be performed by an arbitrary method such as recording in the database in advance and reading from the database.
The left / right direction determination cost C ₁ (k, r, θ _j ) is obtained by Expression (8).

The left-right direction determination cost C ₁ (k, r, θ _j ) is an index representing the proximity between the utterance direction that is the determination target and each utterance direction θ _j prepared in advance. This means that the direction of the utterance is close to θ _j . That is, it is possible to estimate the utterance direction as a determination target by extracting θ _j having the lowest cost from each θ _j prepared in advance.

The speech direction determination unit 106 determines whether θ _j having the minimum left-right direction determination cost C ₁ (k, r, θ _j ) corresponds to the left direction or the right direction with respect to the microphone array 101 and determines the result. Is output (S7). For example, the model average for two orientations θ ₁ = −90 ° and θ ₂ = + 90 °, where the front direction is 0 °, the left direction is a negative angle, and the right direction is a positive angle with respect to the microphone array 101 from the speech position r. If prepared first eigenvector of the left and right orientation determination cost _{C 1 (k, r, θ} 1) <C 1 (k, r, θ 2) at an angle leftward (cost is small theta ₁ is negative when it is Therefore, when C ₁ (k, r, θ ₁ )> C ₁ (k, r, θ ₂ ), it is determined that the direction is right (because θ ₂ having a low cost is a positive angle).

  As described above, the utterance direction estimation apparatus according to the first embodiment can estimate whether a speaker utters leftward or rightward with respect to the microphone array. In addition, since the microphone array may be configured in a form in which a small number of microphones are densely packed, it is possible to configure the microphone array compactly without enclosing the speaker. In addition, since the reverberation is actively used, it is possible to appropriately estimate the utterance direction even in a reverberation environment where the reverberation time is 250 msec or more. In addition, since the eigenvalue decomposition process, which is the core of the process in the present invention, has a small amount of calculation, it is advantageous when it is incorporated in equipment with low CPU specifications such as a portable terminal.

[Second Embodiment]
In the first embodiment, it is determined whether the utterance direction is leftward or rightward. However, the second embodiment further includes a front direction, and the utterance direction is frontal, leftward, or rightward. It is possible to determine whether or not.

<Principle>
As mentioned in the explanation of the principle of the first embodiment, strong reflected waves having directivity with respect to the microphone array are mixed in the initial reverberation time zone, and the power of the reflected waves changes depending on the direction of speech. Specifically, since the direct wave power increases as the utterance direction is the front direction, the reflected wave power decreases, and the direct wave power decreases as it is in the horizontal direction. The power of will increase.

In the case of facing front in FIG. 2, since many direct waves reach the microphone array and the arrival rate of the reflected waves is relatively low, the acoustic propagation vectors representing the direct waves are in the acoustic propagation vector group representing the reflected waves. Compared with greater power. At this time, the first eigenvalue λ ₁ of the correlation matrix is significantly larger than the second eigenvalue λ ₂ and the third eigenvalue λ ₃ . On the other hand, in the case of the horizontal orientation, the direct waves that reach the microphone array are reduced, and more reflected waves reach accordingly. Therefore, the power of the acoustic propagation vector that expresses the direct wave decreases, and the power of the acoustic propagation vector group that expresses the reflected wave increases. The first eigenvalue lambda ₁ when this is smaller than the front direction, the second eigenvalue lambda _{2 Conversely,} the third eigenvalue lambda ₃ is larger than that of the front facing. FIG. 7 shows an image of the difference that occurs in each eigenvalue between the case of facing forward and the case of facing sideways. As described above, since the degree of arrival of the direct wave appears prominently in the eigenvalue λ _i (especially the first eigenvalue λ ₁ ) of the correlation matrix, by evaluating the value taken by the eigenvalue λ _i, Can be carved.

<Configuration>
FIG. 8 shows a functional configuration example of the speech direction estimating apparatus 200 of the present invention, and FIG. 9 shows a processing flow example thereof.

  The utterance direction estimation apparatus 200 includes a microphone array 101 including M (M is an integer of 2 or more) microphones 101-1 to 101-M, an AD converter 12, a frequency domain converter 13, and a correlation matrix calculator. 102, eigenvalue decomposition unit 201, first eigenvector averaging processing unit 104, left-right direction cost calculation unit 105, eigenvalue averaging processing unit 202, front / sideways cost calculation unit 203, speech direction determination unit 204, Is provided. Among these, except for the eigenvalue decomposition unit 201, the eigenvalue averaging processing unit 202, the front / side orientation determination unit 203, and the speaker orientation determination unit 204, the same name / symbol described in the first embodiment is attached. Since it is the same as the element, description of the function / process is omitted.

The eigenvalue decomposition unit 201 performs the same decomposition process as the eigenvalue decomposition unit 103 of the first embodiment, and then outputs a first eigenvector vV ₁ (ω, k) and each eigenvalue λ _i (ω, k). (I = 1, 2,..., M) is normalized by equation (9), and a normalized eigenvalue nλ _i (ω, k) is output (S11).

When calculating the front / side determination cost based only on the first eigenvalue λ ₁ (ω, k) that is the maximum eigenvalue, only the normalized first eigenvalue nλ ₁ (ω, k) is calculated and output. It is good as well.

The eigenvalue averaging processing unit 202 averages the frequencies of the normalized eigenvalues nλ _i (ω, k) obtained for each frequency ω according to the equation (10), and outputs the averaged eigenvalues aλ _i (k) (S12). ).

Here, F ₁ is the frequency index used for averaging, | F ₁ | is the total number of frequency indexes, and F ₁ is appropriately set so as to satisfy Ω ≧ | F ₁ |. When the front / side orientation determination cost is calculated based only on the first eigenvalue λ ₁ (ω, k), which is the maximum eigenvalue, only the averaged first eigenvalue aλ ₁ (k) may be output.

The front / lateral cost calculation unit 203 averages the eigenvalue sequence vaλ (k) = [aλ ₁ (k), aλ ₂ (k),..., Aλ _M (k) obtained by the eigenvalue averaging processing unit 202. ] And a model-averaged eigenvalue sequence vaQ (k, r, θ _j ) = prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the speech position r = From [aQ ₁ (k, r, θ _j ), aQ ₂ (k, r, θ _j ),..., AQ _M (k, r, θ _j )], the front and side directions for each utterance direction θ _j The determination cost C ₂ (k, r, θ _j ) is calculated and output (S13). Here, the model average eigenvalue _{aQ i (k, r, θ} j) as is shown in FIG. 10 for example, the voice signal uttered for each orientation theta _j in speech position r under the same configuration as FIG. 8 On the other hand, it can be obtained by performing the processing up to the eigenvalue averaging processing unit 202. Note that the model averaged eigenvalue aQ _i (k, r, θ _j ) may be input to the front / sideways cost calculation unit 203 by an arbitrary method such as recording in a database in advance and reading from the database.
The front / side orientation determination cost C ₂ (k, r, θ _j ) is obtained by the equation (11).

Note that the eigenvalue difference between the case of speaking in front and the case of speaking in side is reflected particularly prominently in the first eigenvalue, so the front / side determination cost C ₂ (k, r, θ _j ) is You may obtain | require by Formula (12) only from a 1st eigenvalue.

Utterance orientation determining unit 204, the left and right orientation determining cost C ₁ for each _{θ j (k, r, θ} j) and a front-transverse determined cost _{C 2 (k, r, θ} j) is the sum of the C (k , r, θ _j ), the minimum of all combinations of the sums of the left-right determination costs C ₁ (k, r, θ _j ) and the front / side determination costs C ₂ (k, r, θ _j ) It is determined whether θ _j that is the utterance direction of C (k, r, θ _j ) closest to the value corresponds to the front, left, or right direction with respect to the microphone array, and a determination result is output. (S14).

For example, the model first eigenvector and the model eigenvalue are set to θ ₁ = 0 ° (front direction), θ ₂ = −90 ° (left direction), and θ ₃ = + 90 ° (right direction) from the speech position r to the microphone array 101. Consider the case where three orientations are prepared. In this case, three costs of C ₁ (k, r, θ ₁ ), C ₁ (k, r, θ ₂ ), and C ₁ (k, r, θ ₃ ) are output from the left-right cost calculation unit 105. Also, three costs of C ₂ (k, r, θ ₁ ), C ₂ (k, r, θ ₂ ), and C ₂ (k, r, θ ₃ ) are also output from the front / lateral cost calculation unit 203. . The speech direction determination unit 204 receives these as inputs, and C (k, r, θ _j ) = C ₁ (k, r, θ _j ) + C ₂ (k, r, θ _j ) ₁ ), C (k, r, θ ₂ ), and C (k, r, θ ₃ ), respectively. The three cost C obtained (k, r, θ _j) of the minimum cost min {C (k, r, θ j)} a theta _j of estimating the orientation of the utterance to be determined. In this example, if C (k, r, θ ₁ ) is the minimum cost, it faces forward, if C (k, r, θ ₂ ) has the minimum cost, it faces left, C (k, r, θ ₃ ) Can be estimated to be right-facing.

  Thus, according to the utterance direction estimation device of the second embodiment, in addition to the effects of the configuration of the first embodiment, a section called front direction is further provided, and the utterance direction is front direction, left direction, or right direction. Therefore, communication with the other party via the network can be performed more smoothly.

  When the configuration of the utterance direction estimation device of each of the above embodiments is realized by a computer, the processing contents of the functions that each device should have are described by a program. The processing functions are realized on the computer by executing the program on the computer. In this case, at least a part of the processing content may be realized by hardware.

  Further, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

[Service application example]
FIG. 11 shows a configuration example of a service in which the present invention is incorporated into an audio conference terminal. Assume that the conference hall A and the conference hall B are connected by a voice terminal through a network. Speech direction information is extracted from a voice signal picked up by a microphone attached to the voice conference terminal, and transmitted to the other party along with the voice information. By presenting the utterance direction information as visual information on the other party side, it is possible to convey the situation of the place that is difficult to convey only with the voice information.

  In addition, the utterance direction estimation technology can be applied to a minutes system that records the state of communication in a meeting using video and audio. In other words, the utterance direction estimation technique makes it possible to tag the recorded voice and video as to who spoke to whom, which is useful for organizing the minutes.

  In addition, images for services that detect faces from images, for example, surveillance used for surveillance cameras and intercoms, security purposes, and services that determine whether you are paying attention to advertising with digital signage, etc. It is possible to replace the detection of the direction by the detection with an audio signal.

Claims (5)

An analog-to-digital conversion unit that converts analog audio signals collected by a microphone array composed of M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain converter for converting each of the digital audio signals from the time domain to the frequency domain;
A correlation matrix calculator that generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix that is a diagonal matrix having the squares of M eigenvalues as diagonal elements and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues. An eigenvalue decomposition unit that outputs an eigenvector corresponding to the largest eigenvalue among eigenvalues (hereinafter referred to as “first eigenvector”);
A first eigenvector averaging processing unit that outputs an averaged first eigenvector by taking a frequency average for the first eigenvector obtained for each frequency;
From the averaged first eigenvector and the model averaged first eigenvector prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, A left / right cost calculator that calculates and outputs a direction determination cost for each utterance direction θ _j , and
An utterance direction determination unit that determines whether θ _j having the smallest left-right direction determination cost corresponds to the left direction or the right direction with respect to the microphone array and outputs a determination result;
An utterance direction estimation device comprising: In the utterance direction estimation device according to claim 1,
The averaged first eigenvector is obtained by calculating a feature value representing similarity of acoustic propagation characteristics independent of frequency for each of the M elements constituting the first eigenvector, and then taking a frequency average for the feature value. The utterance direction estimation device obtained by An AD conversion step for converting analog audio signals collected by a microphone array comprising M microphones (M is an integer of 2 or more) at a position r into a digital audio signal;
A frequency domain transforming step for transforming each digital audio signal from the time domain to the frequency domain;
A correlation matrix calculation step of generating and outputting an M × M correlation matrix representing a correlation between the digital audio signals converted into the frequency domain;
The correlation matrix is decomposed into an eigenvalue matrix, which is a diagonal matrix having the squares of M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues. An eigenvalue decomposition step for outputting an eigenvector corresponding to the largest eigenvalue (hereinafter referred to as “first eigenvector”);
A first eigenvector averaging process step of outputting an averaged first eigenvector by taking a frequency average for the first eigenvector obtained for each frequency;
From the averaged first eigenvector and the model averaged first eigenvector prepared in advance for each of a plurality of speech directions θ _j (j = 1, 2,..., N, N ≧ 2) at the position r, the left-right direction A left-right cost calculation step for calculating and outputting a determination cost for each utterance direction θ _j ;
An utterance direction determination step of determining whether θ _j having the smallest left-right direction determination cost corresponds to the left direction or the right direction with respect to the microphone array, and outputs a determination result;
Utterance direction estimation method. In the speech direction estimation method according to claim 3,
The averaged first eigenvector is obtained by calculating a feature value representing similarity of acoustic propagation characteristics independent of frequency for each of the M elements constituting the first eigenvector, and then taking a frequency average for the feature value. The utterance direction estimation method obtained by   A program for causing a computer to function as the apparatus according to claim 1.

Images