JP2010103617A - Speech direction estimation device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech direction estimation device which does not request to arrange many microphones so as to enclose a speaker, and can appropriately estimate the speech direction even under environment in which a reverberation time is long. <P>SOLUTION: The plurality of microphones pick up a sound signal originated by the speaker around a microphone array which is composed of the plurality of microphones. A correlation matrix which represents correlation between voice signals each picked up by each microphone, is created, and it is estimated that the speaker has spoken from what direction to the microphone array from an eigenvector obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for estimating the utterance direction of a speaker from an audio signal input to a microphone.

  A system for exchanging voice information such as a telephone or a voice conference terminal is generally called a voice communication system. In the video conference system, the video is added to the audio information and presented, so that the situation of the place is easily transmitted, but in the audio communication system, it is difficult to grasp the situation of the other party. One of the information on the other party's situation is utterance direction information. By receiving this information from the other party, it is possible to grasp the direction in which the speaker is speaking and to facilitate communication.

  Conventional techniques for estimating such speech direction information are disclosed in Non-Patent Documents 1 and 2 and the like, and a configuration example is shown in FIG. The speech direction estimation apparatus 10 in this configuration example estimates speech direction information as follows.

(i) The voice from the speaker 1 is picked up using M (M is an integer of 2 or more) microphones 11-1,. The collected analog signal is converted into a digital signal vx (t) = [x ₁ (t),..., X _M (t)] ^T by the AD converter 12. Here, t represents an index of discrete time.

(ii) The frequency domain transform unit 13 receives a set (frame) of the digital signals composed of a plurality of samples as an input, and performs frequency domain signal VX (ω, n) = [X ₁ (ω, n) by fast Fourier transform or the like. _{, ···, X M (ω,} n)] to convert to ^T. Here, ω represents a frequency index, and the total number of frequency indexes is Ω. N represents the index of the frame.

(iii) The fixed beamformer design unit 14 sets the fixed beamformer VG (ω, r, θ) = [G ₁ (ω, r, θ),..., G _M (for each speaker position and direction. ω, r, θ)] ^T is designed. G _i (ω, r, θ) is a coefficient to be multiplied by the frequency component x _i (ω, n) of the i-th microphone in order to emphasize / suppress the sound source at the speaker position r and the speech direction θ.
In designing, acoustic propagation characteristics between a sound source and a microphone VH (ω, r, θ) = [H ₁ (ω, r, θ),..., H _{M for} each predetermined speaker position and direction. (ω, r, θ)] ^T is obtained using simulation values or actual measurement values. Here, H _i (ω, r, θ) represents an acoustic propagation characteristic between the sound source at the speaker position r and the speech direction θ and the i-th microphone.
The fixed beamformer VG (ω, r, θ) is designed as a value that satisfies the expressions (1) and (2) representing the relationship with the acoustic propagation characteristics.
VH (ω, r _T , θ _T ) ^H · VG (ω, r _T , θ _T ) = 1 (1)
VH (ω, r _U , θ _U ) ^H · VG (ω, r _T , θ _T ) = 0 (2)
Expressions (1) and (2) emphasize the output power of the speaker position r _T and the speech direction θ _T and suppress the output power of the other speaker positions r _U and the speech direction θ _U. It shows that VG (ω, r, θ) is designed.

(iv) In the product-sum calculation unit 15, the frequency domain signal VX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T and the fixed beamformer VG (ω , r, θ) = [G ₁ (ω, r, θ),..., G _M (ω, r, θ)] ^T as an input, for each frequency ω, speaker position r, and speech direction θ. Multiplying the frequency component X _i (ω, n) corresponding to each microphone by the coefficient G _i (ω, r, θ) of the fixed beamformer, and adding the obtained M components, the output Y (ω, n, r, θ) is calculated. This calculation is synonymous with calculating Y (ω, n, r, θ) = VG (ω, r, θ) ^H · VX (ω, n).

(v) The power calculator 16 calculates and outputs power | Y (ω, n, r, θ) | ² from the output Y (ω, n, r, θ) from the product-sum calculator 15.

(vi) The frequency averaging processing unit 17 averages the power | Y (ω, n, r, θ) | ² output from the power calculation unit 16 by frequency, and calculates AY (n, r, θ). obtain. In this calculation, if F ₀ is defined as the frequency index used in the averaging process, and | F ₀ | is defined as the total number of frequency indexes,
Is equivalent to calculating Note that F ₀ satisfies Ω ≧ | F ₀ |.
(vii) The sound source direction selection unit 18 searches for the speaker position r and the speech direction θ where the power AY (n, r, θ) averaged by frequency for each frame is maximum, and the power AY (n , r, θ) is determined as the estimated speech direction θ _out (n).
Hiroshi Nakajima, “Extended Beamforming for Estimating Sound Source Direction”, Proceedings of the Acoustical Society of Japan, September 2005, p.619-620 Hiroshi Nakajima and 8 others, “Sound source directivity estimation using extended beamforming”, Acoustical Society of Japan Proceedings, September 2005, p.621-622

The following two points can be cited as problems of the prior art.
(i) To deal with utterances at any position and to estimate the direction of utterance with high accuracy, a large number of microphones are required and the position of the microphones must be devised. As the output power | Y (ω, n, r, θ) | ² of the fixed beamformer designed for each position and speech direction is different, the speech direction can be estimated with higher accuracy. However, assuming a sound source with a strong directivity in front of the mouth, such as a sound wave radiated from the mouth of the speaker, as shown in FIG. 16, it is necessary to collect sound so as to surround the speaker with a large number of microphones. Depending on the speaker position / speech direction, there is no difference in the output power of the fixed beamformer, and the speech direction estimation error increases (for example, in the experiment of Non-Patent Document 2, 64 microphones are used). Therefore, in order to reduce the error, a large number of microphones are required, the apparatus becomes large, and it is difficult to use it by attaching it to a portable apparatus such as a telephone or an audio conference terminal.

(ii) High reverberation direction estimation performance cannot be obtained in a reverberant environment where the reverberation time (the time from the direct wave arrival time to the 60 dB attenuation from the direct wave pickup power) is 250 msec or more. Below, since many strong reflected waves are mixed, it is difficult to design the acoustic propagation characteristics VH (ω, r, θ) with high accuracy. Therefore, ambiguity occurs in the output of the fixed beamformer, and the estimation accuracy deteriorates. For example, in an actual environment room that is not subjected to low reverberation processing, reverberation time is generally about 250 to 500 msec, so that accurate estimation is difficult.

  An object of the present invention is to estimate the speech direction, which does not require a large number of microphones to be placed so as to surround the speaker, and can appropriately estimate the speech direction even in a reverberant environment with a reverberation time of 250 msec or more. To provide an apparatus, a method, and a program.

  The speech direction estimating apparatus according to the present invention collects a speech signal uttered by a speaker around a microphone array composed of a plurality of microphones by the plurality of microphones, and correlates the speech signals collected by the microphones. A correlation matrix to be expressed is generated, and from which eigenvector obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix, it is estimated in what direction the speaker speaks with respect to the microphone array.

  According to the speech direction estimation apparatus of the present invention, it is not necessary to arrange a large number of microphones so as to surround a speaker, and it is possible to appropriately estimate the speech direction even in a reverberant environment with a reverberation time of 250 msec or more. It becomes.

[First Embodiment]
FIG. 1 shows a functional configuration example of the speech direction estimating apparatus 100 of the present invention, and FIG. 2 shows a processing flow example thereof. The utterance direction estimation apparatus 100 estimates whether the utterance direction is leftward or rightward with respect to the microphone array.

  The utterance direction estimation apparatus 100 includes a microphone array 101 including M microphones (M is an integer of 2 or more) 101-1 to 101-M, an AD conversion unit 12, a frequency domain conversion unit 13, and a correlation matrix calculation unit. 102, an eigenvalue decomposition unit 103, a first eigenvector power calculation unit 104, a first frequency averaging processing unit 105, and a left / right direction determination unit 106. Among them, the AD conversion unit 12 and the frequency conversion unit 13 are the same as those used in the speech direction estimation apparatus 10 described in the background art.

  In the prior art, as shown in FIG. 16, it was necessary to arrange a large number of microphones so as to surround the speaker. However, in the present invention, M microphones 101-1 to 101-M can be provided. It is sufficient to arrange them closely. Although the number of microphones constituting such a microphone array 101 is not too large, according to the configuration of the present invention described below, it is possible to estimate the utterance direction with two or more microphones. The arrangement may be two-dimensional or three-dimensional. By adopting a configuration in which a small number of microphones are arranged densely in this way, it can be attached to a portable device such as a telephone or an audio conference terminal, and the direction of speech by surrounding speakers can be estimated It becomes. The speaker speaks around the microphone array 101. FIG. 3 shows an image of a speaker talking around a microphone array 101 composed of seven microphones as viewed from above, and the direction of the arrow is the direction of speech. FIG. 3 (a) shows a state where a speaker is speaking in a front direction at each position, and FIG. 3 (b) shows a state where a speaker is speaking in a horizontal direction.

The AD conversion unit 12 converts the analog voice signals uttered by the speaker 1 collected by the M microphones 101-1 to 101 -M into digital voice signals x ₁ (t),..., X _M (t (S1). Here, t represents an index of discrete time.

The frequency domain transform unit 13 receives as input a set (frame) of the above digital speech signals composed of a plurality of discrete time samples, and performs frequency domain digital speech signals X ₁ (ω, n),. It is converted into X _M (ω, n) and output (S2). Here, n represents a frame index, and ω represents a frequency index. The total number of frequency indexes is Ω.

The correlation matrix calculation unit 102 receives the digital audio signals X ₁ (ω, n),..., X _M (ω, n) in the frequency domain as inputs, and an M × M correlation matrix R representing the correlation between the signals. (ω, k) is sequentially generated and output for each frequency ω by equation (3) (S3).
R (ω, k) = E [VX (ω, n) ・ VX ^H (ω, n)] (3)
Here, VX (ω, n) = [X ₁ (ω, n),..., X _M (ω, n)] ^T
Note that H represents conjugate transposition, and E calculates VX (ω, n) · VX (ω, n) ^H for each frame, and then calculates an expected value for each L frame by averaging processing or the like. It is an operator. That is, the correlation matrix is sequentially output at a rate of once per L frame, and k represents an output index of the correlation matrix. L is preferably an integer greater than or equal to M.

The eigenvalue decomposition unit 103 receives the correlation matrix R (ω, k) as input, and first, M eigenvalues λ ₁ (ω, n),..., Λ _M (ω, n) so as to satisfy Equation (4). ) Eigenvalue matrix Λ (ω, k), which is a diagonal matrix with each square as a diagonal element, and M eigenvectors vv ₁ (ω, n),..., Vv _M (ω, k) Is decomposed into eigenvector matrix V (ω, k) by the eigenvalue decomposition method.
R (ω, k) = V (ω, k) ・ Λ (ω, k) ・ V ^H (ω, k) (4)
Where Λ (ω, k) = diag [λ ₁ ² (ω, k),..., Λ _M ² (ω, k)]
λ ₁ (ω, k) ≧ λ ₂ (ω, k) ≧ ・ ・ ・ ≧ λ _M (ω, k)
V (ω, k) = [vv ₁ (ω, k),..., Vv _M (ω, k)] ^T
vv _i (ω, k) = [v _{i, 1} (ω, k),..., v _{i, M} (ω, k)]
Note that diag [•] is an operator having the components in [•] as elements of a diagonal matrix.
Then, the first eigenvector vv ₁ (ω, k) corresponding to the first eigenvalue λ ₁ (ω, k) which is the maximum eigenvalue is output (S4).

The first eigenvector power calculation unit 104 receives the first eigenvector vv ₁ (ω, k) as an input, and v _1,1 (ω, k),... Constituting the first eigenvector vv ₁ (ω, k). For M elements of v _{1, M} (ω, k), the power is calculated by the equation (5), respectively, and M power elements pv _1,1 (ω, k),..., pv _{1, M} (ω, k) is output (S5).
pv _{1, i} (ω, k) = | v _{1, i} (ω, k) | (5)

The first frequency averaging processing unit 105 formulas _M power elements pv _1,1 (ω, k),..., Pv _{1, M} (ω, k) generated for each frequency ω, respectively. The average value is calculated according to (6) and M averaged power elements apv _1,1 (k),..., Apv _{1, M} (k) are output (S6).
F ₁ is an index of frequencies used for averaging, | F ₁ | is a total number of frequency indexes, and F ₁ is appropriately set so as to satisfy Ω ≧ | F ₁ |.

The left / right direction determination unit 106 receives M averaged power elements apv _1,1 (k),..., Apv _{1, M} (k) as input, and determines whether the left direction is spoken or the right direction is spoken. The result is output (S7). The left-right orientation determination is performed by using two averaged power elements apv _{1, α} (k), apv corresponding to two microphones 101-α and 101-β among the M microphones constituting the microphone array 101. This is done by taking a ratio of _{1, β} (k) and comparing it to a predetermined threshold value thr1. FIG. 4 illustrates an example of the determination image in the horizontal direction. In this example, the case where the speech is directed toward the midpoint between the microphone α and the microphone β shown in FIG. 4A is apv _{1, α} (k) / apv _{1, β} (k) = thr1, and this direction is set as follows. The reference speech direction in the left-right direction is determined based on the magnitude relationship between apv _{1, α} (k) / apv _{1, β} (k) and thr1. Specifically, since the rate of attenuation of apv _{1, β} (k) is larger than apv _{1, α} (k) as it goes to the left, apv _{1, α} (k) / apv _{1, β} (k )> Thr1 can be determined to be leftward (FIG. 4 (b)), and the more rightward is, the greater the rate at which apv _{1, α} (k) attenuates than apv _{1, β} (k). Therefore, when apv _{1, α} (k) / apv _{1, β} (k) <thr1, it can be determined to be rightward (FIG. 4 (c)). Note that the two microphones have the widest left-right spacing with respect to the speaker's position so that the average power elements apv _{1, α} (k) and apv _{1, β} (k) tend to differ. It is desirable to choose.

  A theoretical background capable of determining the left-right direction of speech with such a configuration will be described. FIG. 5 shows the propagation characteristics of the audio signal in the time domain. Propagation characteristics can be broadly divided into three types: direct wave, initial reverberation, and rear reverberation. In the time zone in which direct wave and initial reverberation are observed, the directivity with respect to the microphone array composed of a plurality of microphones. It is known that waves with In particular, in the initial reverberation time zone (after the arrival of the direct wave, the time from the direct wave pickup power to the attenuation of 10 dB), a strong reflected wave with directionality is mixed, but the power of this reflected wave depends on the direction of speech. Change. Specifically, since the direct wave power increases as the utterance direction is the front direction, the reflected wave power decreases, and the direct wave power decreases as it is in the horizontal direction. The power of will increase. The present invention uses such a property to estimate the speech direction.

This will be described below in accordance with the configuration of the present invention. FIG. 6 shows how the utterance state in front and side affects each eigenvalue λ _i (ω, k) of the correlation matrix R (ω, k). Here, a case where a microphone array is configured by three microphones is illustrated. When facing the front, many direct waves reach the microphone array, and the arrival rate of the reflected waves is relatively low. Therefore, as shown in FIG. Compared to the basis vector group to be expressed, it has a large power. At this time, the first eigenvalue λ ₁ (ω, k) is significantly larger than the second eigenvalue λ ₂ (ω, k) and the third eigenvalue λ ₃ (ω, k). On the other hand, in the case of the horizontal orientation, the direct waves that reach the microphone array are reduced, so that more reflected waves arrive accordingly. For this reason, as shown in FIG. 6B, the power of the basis vectors expressing the direct wave decreases, and the power of the basis vector group expressing the reflected wave increases. At this time, the first eigenvalue λ ₁ (ω, k) is smaller than that in the front direction, and conversely, the second eigenvalue λ ₂ (ω, k) and the third eigenvalue λ ₃ (ω, k) are in the front direction. Larger than the case. FIG. 7 shows an image of the difference that occurs in each eigenvalue between the case of facing forward and the case of facing sideways.

  From the above, it can be seen that the power of the basis vector expressing the direct wave is significantly reflected in the first eigenvalue. If so, the strength of each power element corresponding to the M microphones of the first eigenvector corresponding to the first eigenvalue is how strong the direct wave reaches each of the M microphones. It can be considered as a measure of The power at which direct waves reach each microphone varies depending on the direction of speech. Therefore, as described above, the ratio of the power elements of any two microphones in the reference utterance direction is set as a threshold (thr1), and the threshold and the power elements of the two microphones in a certain utterance direction It is possible to determine whether the utterance is uttered in the left direction or the utterance in the right direction with respect to the reference utterance direction from the magnitude relationship.

  As described above, according to the utterance direction estimation apparatus of the first embodiment, since a small number of microphones need only be densely arranged, it is possible to make a compact configuration without enclosing a speaker with a large number of microphones. Become. In addition, since the reverberation is actively used, it is possible to appropriately estimate the utterance direction even in a reverberation environment where the reverberation time is 250 msec or more. In addition, since the eigenvalue decomposition process, which is the core of the process in the present invention, has a small amount of calculation, it is advantageous when it is incorporated in equipment with low CPU specifications such as a portable terminal.

[Second Embodiment]
In the first embodiment, it is determined whether the utterance direction is leftward or rightward. However, the second embodiment further includes a front direction, and the utterance direction is frontal, leftward, or rightward. It is possible to determine whether or not.

FIG. 8 shows a functional configuration example of the speech direction estimating apparatus 200 of the present invention, and FIG. 9 shows a processing flow example thereof.
The utterance direction estimation apparatus 200 includes a microphone array 101 including M (M is an integer of 2 or more) microphones 101-1 to 101-M, an AD converter 12, a frequency domain converter 13, and a correlation matrix calculator. 102, an eigenvalue decomposition unit 201, a first eigenvector power calculation unit 104, a first frequency averaging processing unit 105, a left / right orientation determination unit 106, a second frequency averaging processing unit 202, and a front / side orientation determination unit 203. Of these components, the components other than the eigenvalue decomposition unit 201, the second frequency averaging processing unit 202, and the front / side orientation determination unit 203 are the same as the components having the same names and symbols described in the first embodiment. Therefore, description of functions and processes is omitted.

The eigenvalue decomposition unit 201 performs the same processing as the eigenvalue decomposition unit 103 of the first embodiment, further normalizes the first eigenvalue λ ₁ (ω, k) by the equation (7), and obtains the first normalized eigenvalue nλ _1. (ω, k) is output (S8).

The second frequency averaging processing unit 202 calculates the average value of the first normalized eigenvalue nλ ₁ (ω, k) obtained for each frequency ω by the equation (8), and the first averaged eigenvalue aλ ₁ (k) is output (S9).

F ₂ is an index of frequencies used for averaging, | F ₂ | is the total number of frequency indexes, and F ₂ is appropriately set so as to satisfy Ω ≧ | F ₂ |.

The front / horizontal direction determination unit 203 receives the determination result from the left / right direction determination unit 106 and the first averaged eigenvalue aλ ₁ (k), and uses the first averaged eigenvalue aλ ₁ (k) as a predetermined threshold value thr2. If aλ ₁ (k) <thr2, the speaker determines that the speaker is facing sideways with respect to the microphone array, and otherwise determines that the speaker is facing frontward. If it is determined to be in the horizontal direction, the determination result in the left-right direction determination unit 106 is output as it is, and if not, the determination result that it is in the front direction is output (S10). FIG. 10 shows an example of the front / side orientation determination image. Here, thr2 may be arbitrarily set according to the environment and the position of the speaker. Since the first averaged eigenvalue aλ ₁ (k) is obtained for each frame group k, the determination result is also output for each frame group k.

  A theoretical background that can determine whether the speech direction is the front direction or the horizontal direction with such a configuration will be described. As described in the first embodiment, the power of the basis vector representing the direct wave is significantly reflected in the first eigenvalue. Specifically, in the case of the front direction, the power of the base vector expressing the direct wave shows a large value and the first eigenvalue also shows a large value. The first eigenvalue is also smaller than when facing the front. Therefore, by using the first eigenvalue as a front / side determination parameter, it is possible to appropriately determine that the first eigenvalue is front-facing if it is larger than a certain threshold value and that it is lateral if it is smaller.

  Thus, according to the utterance direction estimation device of the second embodiment, in addition to the effects of the configuration of the first embodiment, a section called front direction is further provided, and the utterance direction is front direction, left direction, or right direction. Therefore, communication with the other party via the network can be performed more smoothly.

  When the configuration of the utterance direction estimation device of each of the above embodiments is realized by a computer, the processing contents of the functions that each device should have are described by a program. The processing functions are realized on the computer by executing the program on the computer. In this case, at least a part of the processing content may be realized by hardware.

  Further, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

[Verification of effects]
In the sound collection environment shown in FIG. 11 (a), the effect of the present invention was verified under the conditions shown in FIG. 11 (b). The definition of the utterance direction is as shown in FIG.

FIG. 12 shows the relationship between each utterance direction and the first to seventh averaged eigenvalues aλ _i (k). Here, the second to seventh averaged eigenvalues are obtained by the same method as the first averaged eigenvalue. FIG. 12A shows the case where the reverberation time is 250 msec, and FIG. 12B shows the case where the reverberation time is 400 msec. In any reverberation time, the first averaged eigenvalue varies greatly depending on the speech direction, but the second to seventh averaged eigenvalues have a small difference depending on the speech direction. The first eigenvalue is the largest when it is 0 ° (frontward) and the smallest when it is ± 90 ° (laterally). From this verification result, it can be seen that the configuration of the present invention for determining the front / sideways orientation based on the first averaged eigenvalue shown in the second embodiment is valid and effective.

  FIG. 13 shows a comparison between the left and right utterance direction estimated by the configuration of the first embodiment and the actual utterance direction. FIG. 13A shows a case where the reverberation time is 250 msec, and FIG. 13B shows a case where the reverberation time is 400 msec. In any reverberation time, a correct answer rate of 75% or more was obtained. From this verification result, it can be seen that the configuration of the present invention for determining the left-right direction from the two power elements of the first eigenvector shown in the first embodiment is valid and effective.

[Service application example]
FIG. 14 is a structural example of a service in which the present invention is incorporated in an audio conference terminal. Assume that the conference hall A and the conference hall B are connected by a voice terminal through a network. Speech direction information is extracted from a voice signal picked up by a microphone attached to the voice conference terminal, and transmitted to the other party along with the voice information. By presenting the utterance direction information as visual information on the other party side, it is possible to convey the situation of the place that is difficult to convey only with the voice information.

The figure which shows the function structural example of the speech direction estimation apparatus of 1st Embodiment. The figure which shows the example of a processing flow of the speech direction estimation apparatus of 1st Embodiment. The image figure which shows the positional relationship of a microphone, a speaker, and a speech direction. The image figure about the method of determining left-right orientation. The figure which shows the propagation characteristic of an audio | voice signal in a time domain. The image figure which shows the correlation with an utterance direction, a base vector, and an eigenvalue / eigenvector. The image figure which shows the relationship between an utterance direction and an eigenvalue. The figure which shows the function structural example of the speech direction estimation apparatus of 2nd Embodiment. The figure which shows the example of a processing flow of the utterance direction estimation apparatus of 2nd Embodiment. The image figure about the method of judging front direction and sideways. The figure which shows the verification environment and conditions of an effect. The figure which shows the verification result about the relationship between an utterance direction and an eigenvalue. The figure which shows the verification result of left-right direction estimation. The figure which shows the service structural example which incorporated this invention in the audio conference terminal. The figure which shows the function structural example of the speech direction estimation apparatus by a prior art. The image figure which shows the relationship between the microphone and speaker by a prior art.

Claims (9)

  The sound signals spoken by the speaker around the microphone array composed of a plurality of microphones are picked up by the plurality of microphones, and a correlation matrix representing the correlation between the sound signals picked up by the respective microphones is generated. An utterance direction estimation device for estimating a direction in which a speaker speaks in the left-right direction with respect to the microphone array from eigenvectors obtained by decomposing a correlation matrix into an eigenvalue matrix and an eigenvector matrix. In the utterance direction estimation device according to claim 1,
An AD converter for converting analog audio signals picked up by M microphones (M is an integer of 2 or more) into digital audio signals;
A frequency domain transform unit that transforms each of the digital audio signals from a time domain to a frequency domain in units of frames of a plurality of samples;
A correlation matrix calculation unit that sequentially generates and outputs an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain for each frequency as an expected value for each of a plurality of frames;
The correlation matrix is decomposed into an eigenvalue matrix that is a diagonal matrix having the squares of the M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues, and the maximum eigenvalue is obtained. An eigenvalue decomposition unit that outputs a corresponding eigenvector (hereinafter referred to as “first eigenvector”);
A first eigenvector power calculator that calculates power for each of M elements constituting the first eigenvector and outputs M power elements;
A first frequency averaging processor that calculates an average value for each of the M power elements generated for each frequency and outputs M averaged power elements;
Of the M averaged power elements, the ratio of two averaged power elements corresponding to any two microphones is taken and compared with a predetermined threshold value so that the speaker can A left-right direction determination unit that determines whether the utterance is directed leftward or rightward with respect to the array and outputs a result;
An utterance direction estimation device comprising: In the utterance direction estimation apparatus according to claim 1 or 2,
An utterance direction estimation apparatus characterized by further estimating whether or not an utterance direction is a front direction using eigenvalues obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix. In the utterance direction estimation device according to claim 3,
The eigenvalue decomposition unit further normalizes the maximum eigenvalue (hereinafter referred to as “first eigenvalue”), and outputs a first normalized eigenvalue,
A second frequency averaging processing unit that calculates an average value for the first normalized eigenvalue generated for each frequency and outputs the first averaged eigenvalue;
By comparing the first averaged eigenvalue with a predetermined threshold value, it is determined whether the speaker speaks in the front direction or the side direction with respect to the microphone array. A determination result that the utterance is spoken in the direction, and a determination result in the left and right direction determination unit when the determination is made in the horizontal direction, the front and side direction determination unit that outputs the determination result as it is,
An utterance direction estimation apparatus, further comprising:   The sound signals spoken by the speaker around the microphone array composed of a plurality of microphones are picked up by the plurality of microphones, and a correlation matrix representing the correlation between the sound signals picked up by the respective microphones is generated. An utterance direction estimation method for estimating a direction in which a speaker utters in the left-right direction with respect to the microphone array from eigenvectors obtained by decomposing a correlation matrix into an eigenvalue matrix and an eigenvector matrix. In the speech direction estimation method according to claim 5,
AD conversion step for converting analog audio signals picked up by M (M is an integer of 2 or more) microphones into digital audio signals;
A frequency domain transforming step for transforming each digital audio signal from the time domain to the frequency domain in units of frames of a plurality of samples;
A correlation matrix calculating step for sequentially generating and outputting an M × M correlation matrix representing the correlation between the digital audio signals converted into the frequency domain for each frequency as an expected value for each of a plurality of frames;
The correlation matrix is decomposed into an eigenvalue matrix that is a diagonal matrix having the squares of the M eigenvalues as diagonal elements, and an eigenvector matrix composed of M eigenvectors corresponding to the eigenvalues, and the maximum eigenvalue is obtained. An eigenvalue decomposition step of outputting a corresponding eigenvector (hereinafter referred to as “first eigenvector”);
A first eigenvector power calculating step of calculating power for each of M elements constituting the first eigenvector and outputting M power elements;
A first frequency averaging step of calculating an average value for each of the M power elements generated for each frequency and outputting M averaged power elements;
Of the M averaged power elements, the ratio of two averaged power elements corresponding to any two microphones is taken and compared with a predetermined threshold value so that the speaker can A left-right orientation determination step for determining whether the speech is directed leftward or rightward with respect to the array and outputting the result;
A speech direction estimation method characterized in that is executed. In the speech direction estimation method according to claim 5 or 6,
An utterance direction estimation method characterized by further estimating whether or not an utterance direction is a front direction using eigenvalues obtained by decomposing the correlation matrix into an eigenvalue matrix and an eigenvector matrix. In the speech direction estimation method according to claim 7,
In the eigenvalue decomposition step, the maximum eigenvalue (hereinafter referred to as “first eigenvalue”) is further normalized to output a first normalized eigenvalue,
A second frequency averaging step for calculating an average value for the first normalized eigenvalue generated for each frequency and outputting a first averaged eigenvalue;
By comparing the first averaged eigenvalue with a predetermined threshold value, it is determined whether the speaker speaks in the front direction or the side direction with respect to the microphone array. A determination result that the utterance is spoken in the direction, and a determination result in the left-right direction determination step when the determination result in the horizontal direction is output as it is, a front / horizontal determination step;
Is further executed to utterance direction estimation method.   A program for causing a computer to function as the apparatus according to claim 1.