JP2008042334A - Head-set microphone-array audio input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a head-set microphone-array audio input device capable of receiving a necessary signal with an excellent sensibility, even when the device has a superior mounting property and a peripheral environment is placed in an atmosphere having much noises. <P>SOLUTION: The head-set microphone-array audio input device is composed of a head band, storage cases with ear pads fitted to both ends of the head band, respectively, and struts fitted to both storage cases with the ear pads, respectively. An arbitrary number of one or more of microphones are mounted on the struts, and output signals from the microphones can be output to the storage cases with the ear pads. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention suppresses ambient noise as a user voice and enhances the user voice in a real environment where various environmental noises exist, thereby suppressing the ambient noise and enhancing the SNR even in a poor noise environment. The present invention relates to a headset type microphone array voice input device capable of recording user voice with (signal to noise ratio).

Conventionally, in order to record a user voice with a high SNR in a poor noise environment (see, for example, Patent Document 1), a close-talking headset microphone in which one microphone is arranged near the mouth (for example, see Patent Document 2). ) Is used.
In Patent Document 1, in order to obtain a speech estimation method and a speech recognition method that operate robustly even in a speech signal input under noise or a speech signal mixed with noise in a communication path,
An acoustic analysis step that cuts out the input acoustic signal for each short-term segment and performs short-time frequency analysis, an element estimation step that estimates the elements required for speech estimation, and speech using the elements obtained by the element estimation step A speech estimation method including a speech estimation step for estimation. Specifically, the input acoustic signal is cut out for each short-term segment, short-term frequency analysis is performed, the speech envelope that the speech recognition holds in the codebook is used as knowledge, a sound model is generated, and the short-time segment is generated. The spectrum information obtained by frequency analysis is regarded as a probability density function, the mixture weight value is estimated using the maximum posterior probability estimation, and the existence assumption of the element that generates the sound model having the maximum weight value at each time is the most. A method of outputting the element with a high likelihood is disclosed.

However, the technique disclosed in Patent Document 1 cannot be used in an environment where a sufficient amount of information related to transmission path distortion and background noise is not available in advance. Even if such prior information on noise can be used, complicated work such as learning of a model (HMM) is required based on the prior information.
Further, in Patent Document 2, in order to effectively and reliably control a connected headset with a simple and low-cost structure, the mobile terminal device inputs an internal speaker 2 that outputs an acoustic signal and an acoustic signal. An internal microphone 3, a connector 4 to which an external headset 10 including a headset speaker 12 and a headset microphone 13 is connected, a processing circuit 5 that controls the internal speaker 12, the internal microphone 13, and the external headset 10; A switching circuit 6 is provided that switches and connects the processing circuit 5 to the internal speaker 12 and the internal microphone 13 and the external headset 10. The processing circuit 5 is connected to the internal microphone 3 and includes a detection circuit 7 that detects an electrical change, detects whether or not the external headset 10 is connected, and operates the headset 10 according to the detection result. Control.
However, since the thing of patent document 2 has one microphone, the discrimination capability of noise and a required signal is bad, and there exists a fault to which the sensitivity of a received signal is greatly influenced by the direction and position which a microphone faces.
JP 2003-076393 A JP 2002-152365 A

  Conventional headset microphones can be recorded with high SNR even in the presence of some noise in a normal environment by placing a single microphone near the mouth. For example, from a factory or speaker In a severe noise environment where sound is emitted at a high volume, the SNR is lowered and the effect is lost. In such a case, even if a headset microphone is used, it is not only difficult to record a voice with a high SNR in a noisy environment. For example, when a voice recorded with a headset microphone is recognized, ambient noise is reduced. Problems such as misrecognition occur.

  An object of the present invention is to provide a headset type microphone array audio input device that is capable of receiving necessary signals with high sensitivity even in a wearable and ambient environment with a lot of noise. .

In order to improve discrimination from ambient noise, a microphone array structure in which a plurality of microphones for collecting signals is arranged is employed, and special signal processing is performed on the premise of the structure. By using a microphone array, it is possible to perform signal processing such as estimating the sound source location and direction of arrival of sound waves, and suppressing interference noise by emphasizing the target sound, which is often used as a voice input device in a noisy environment. Have the advantage of.
However, the conventional microphone array is not used in such a way that the user himself wears the microphone array, such as being mounted on a microphone stand or being incorporated in a device.
On the other hand, the close-talking headset microphone uses only one microphone, but the SNR of the user voice can be made relatively high with respect to normal ambient noise by bringing the microphone closer to the mouth. The simple principle is used. In the present invention, a microphone array is paired and mounted in a close-talking manner, so that it malfunctions in voice recording and voice recognition with a high SNR even in a poor noise environment, which has both the effect of a headset microphone and the effect of a microphone array. A headset microphone array with a small amount of noise is formed.

Specifically, (1) a headset type microphone array audio input device includes a headband, storage cases with ear pads provided at both ends of the headband, and support columns provided in the storage cases with both ear pads, respectively. Thus, the support column is provided with an arbitrary number of one or more microphones so that an output signal of the microphone can be output to the housing case with the ear pad.
(2) The headset type microphone array audio input device according to the above (1) is characterized in that a plurality of microphones are provided for each of the columns, and they are arranged with a predetermined distance therebetween.
(3) The headset type microphone array audio input device according to the above (1) or (2) is characterized in that a microphone for each of the columns is provided on a surface facing both columns.

(4) In the headset type microphone array audio input device according to any one of (1) to (3), each of the storage cases with ear pads includes a case main body and an ear pad, and an output of each microphone is provided in the case main body. A circuit is provided that outputs an audio signal obtained by arithmetic processing of the signal.
(5) The headset type microphone array audio input device according to any one of (1) to (4), wherein an earphone for inputting the audio signal is provided in each case body, and a speaker is provided in each ear pad. It is characterized by that.
(6) The headset type microphone array audio input device according to any one of (1) to (5) is characterized in that a wireless communication means is provided in each case body.
(7) In the headset type microphone array audio input device according to any one of (1) to (6), the support column is provided on each case body so that the mounting angle can be adjusted.
(8) In the headset type microphone array audio input device according to any one of (1) to (7), the both columns are provided in parallel to each other.
(9) In the headset type microphone array audio input device according to any one of (1) to (8), the headbands have a structure that can slide and change its length. Each case main body is provided at a mounting angle with respect to the headband.

The headset type microphone array audio input device of the present invention is based on the principle of a headset microphone (a configuration in which a plurality of microphones are arranged in an arbitrary number of pairs on the left and right sides by simply mounting the microphone array in a close-talking type. Since the signals (including coordinate position information and the like) collected from these microphones are signal-processed, a necessary signal can be extracted with high accuracy), so that the SNR of the user voice can be increased with respect to ambient noise. Furthermore, by using the microphone array processing to suppress the interference noise and enhance the user voice, voice recording with a higher SNR can be realized. In addition, since the sound source generation position and sound wave arrival direction can be estimated by using the microphone array, the input sound is based on the estimation result of the sound source generation position and sound wave arrival direction even in a poor noise environment. Since it is easy to determine whether the user's voice or ambient noise, misrecognition due to ambient noise can be avoided in applications such as voice recognition.
The thing of patent document 1 cannot be used in the environment where sufficient information regarding a transmission path distortion and background noise is not previously available. Even if such prior information on noise can be used, complicated work such as learning of a model (HMM) is required based on the prior information. In the device of the present invention, by using the microphone array, it is possible to suppress noise placed at spatially different positions even if prior information on noise cannot be used.

  Embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a schematic diagram of a headset type microphone array audio input apparatus according to the present invention. A conventional headset microphone has a structure in which a support is fixed to only one of the left and right sides of the headset, and one microphone is disposed at the tip of the column. On the other hand, the headset type microphone array audio input device of the present invention has a structure in which the headset microphone is fixed to the right and left sides of the headset, and one microphone is arranged at the tip.
The headset 1 of the present invention includes a headband 3 for mounting on the head, storage cases 2R and 2L with ear pads attached to both ends of the headband 3, and a substantially rod-shaped support 4R provided on the storage case 2R with ear pads. And a substantially rod-like support 4L provided in the storage case 2L with ear pads. The storage cases 2R and 2L with ear pads are composed of case bodies 2Ra and 2La and ear pads 2Rb and 2Lb, respectively.
In the columns 4R and 4L, the same number of one or more arbitrary numbers of microphones 5 are arranged separately. Preferably, the same three microphones 5 are provided for each of the columns 4R and 4L.
The headband 3 can be configured to be slidable as described below to enable adjustment of the length.

  The case main body 2Ra and 2La of the earpad-equipped storage cases 2R and 2L store a battery box, a wireless transmission / reception circuit, a processing circuit for performing a necessary process on an input signal from each microphone 5 of the microphone array 6, and the like. To do. The case main bodies 2Ra and 2La and the ear pads 2Rb and 2Lb adjust the distance between each other, for example, by screwing a hollow bolt provided on the ear pad and a nut integrated with the case. You may make it adjust a mutual space | interval by another means.

  In the present invention, the columns 4R and 4L are fixed to both the case bodies 2Ra and 2La of the housing cases 2R and 2L with ear pads, respectively, and the microphone array 6 is formed by arranging a plurality of microphones 5 on the columns. The microphone 5 used for mounting is a very small one having a size of about 5 mm × 3 mm, such as a silicon microphone. The number of microphones arranged on the columns 4R and 4L and the interval between the microphones can be adjusted by software, and are arbitrary. In the microphone array 6, it is often necessary that the relative positional relationship between the microphones 5 is always maintained. However, in the case of a headset, the distance between the left and right microphone arrays 6 may change depending on the size of the head. In order to cope with this, as shown in FIG. 2, the distance between the left and right microphone arrays 6, 6 is the distance between the case bodies 2Ra and 2La of the storage cases 2R and 2L to which the columns 4R and 4L are attached and the ear pads 2Rb and 2Lb. Adjust by adjusting the interval.

FIG. 2 is a block diagram showing the means for adjusting the distance between the storage cases with both ear pads of the present invention.
The distance between the two is adjusted by a structure in which both can be moved in the direction of the arrow in FIG.
In addition to such a structure, the headband 3 supporting the case main bodies 2Ra and 2La provided with the microphone array 6 is formed in a U-shape by a rigid member in order to improve the mounting state. Each of the case main bodies 2Ra and 2La can be configured so as to be swingable in the front-rear direction, and a movable head band formed in a U shape by a rigid member.
Further, in order to make it possible to adjust the pressing of the left and right ears by the pair of left and right storage cases 2R and 2L according to the size of the user's head, the pair of left and right case main bodies 2Ra and 2La are substantially omitted. A pair of arc-shaped left and right headbands 3 are slid and adjusted. The pair of left and right headbands 3 are combined so that they can be adjusted in a nesting manner, and the length of the pair of left and right headbands 3 is adjusted according to the size of the user's head. Align the headphones unit with the left and right ears of the user.
In addition, the case body 2Ra and 2La with respect to the headband are configured so that the attachment angle in an arbitrary direction can be adjusted.
The pair of columns 4R and 4L are arranged in parallel, have a length that extends beyond the user's mouth, and allows the microphone to be arranged thereon.

(Parallel microphone array audio input device)
The voice input unit includes a sound receiving unit including a plurality of microphone arrays 6 arranged to be separated from each other in order to receive a user voice.
The configuration of the parallel microphone array audio input device shown in FIGS. 1 and 2 will be described below. As shown in FIG. 1, the two metal fittings to which the microphones are attached have one end fixed to the headband and, for example, have a length that extends beyond the user's mouth in parallel at an interval of 20 cm. An arbitrary number of, for example, two microphones (four in total) are arranged at an arbitrary interval, for example, an interval of 3 cm.
As shown in FIG. 3, the voice input unit includes parallel microphone arrays 30 a and 30 b, a microphone amplifier, and an ADC (analog / digital converter) 32.
The sound receiving means includes at least a plurality of microphones, and preferably a microphone array in which a large number of microphones are arranged in an array. The microphones are arranged at least apart from each other so that the vectors from the sound sources are different. More preferably, the microphones are disposed on both sides of the user's mouth. Thus, by arrange | positioning on both sides of a user's mouth tip, a user's voice input becomes easy and clear.

FIG. 3 is a block diagram of a processing circuit housed in the case body of the present invention. In particular, it is an example of a processing circuit that performs a necessary process on an input signal from each microphone of the microphone array.
In the processing circuit of the present invention, parallel microphone arrays 30a and 30b are connected to a CPU (Central Processing Unit) board 33 via a microphone amplifier and an ADC 32, and the CPU (Central Processing Unit) board 33 is connected to a storage device 34 by a bus. Connected. A CPU (Central Processing Unit) board 33 is connected to the display 31 for output display, is connected to an earphone speaker 35 in the ear pad, and is further connected to a transmitter 36 in the case main bodies 2Ra and 2La. . The transmission / reception device 36 is not limited to wired or wireless, and any communication means can be employed.
The CPU (central processing unit) board 33 is a board on which a CPU is mounted, and includes an utterance position estimation unit and a control unit. The utterance position estimating means and the control means are constituted by a CPU board 33 and a storage device 34 connected thereto.
The utterance position estimation means estimates the utterance position of the user based on the multichannel audio data received by the parallel microphone arrays 30a and 30b, and outputs a utterance position estimation signal.
The sampling rate can be arbitrarily set, for example, 8 kHz, and the number of quantization bits can be arbitrarily set, for example, 16 bits. When increasing the processing accuracy, the sampling rate and the number of quantization bits are increased.

(Image display means)
The headset-type microphone array audio input device can include a small and thin display (for example, liquid crystal, EL (electroluminescence, plasma display, etc.)), a head-mounted display, etc. as the image display means. The result of the estimation process is shown visually.

この３次元空間中の任意の位置に置かれた音源から出力された音響信号を、３次元空間中の任意の位置に配置されたＱ個のマイクロフォンで受音する。各マイクロフォンの位置は位置ベクトルＰ_ｑで表す。

(Voice recognition device)
FIG. 4 is a block diagram of the speech recognition apparatus.
The voice recognition device 40 includes a microphone array processing unit 41 and a voice recognition processing unit 42.
The microphone array processing unit 41 estimates the direction of sound wave arrival of a sound source at a long distance, which estimates the sound wave arrival direction of the sound source at a long distance from the sound of the microphone array sound input device 43 and the sound output from the device 43. Based on the sound source position information of the means 45, the sound source position estimation means 46 at a short distance and the sound source position information of the means 45 and 46, the output of the device 43 Based on the sound source position information of the sound source separation processing means 44 and means 45 and 46 for separating the sound of the sound source to be extracted from the expanded sound, the utterance of the user (headset type microphone array sound input device wearer) is detected. The voice signal from the sound source separation processing means 44 is switched according to the detection signal from the user's speech detection means 47 and the user's speech detection means 47. Composed of switching means 48 for force.
The voice recognition processing unit 42 performs a feature correction processing unit 49 for correcting a feature on the voice signal from the switching unit 48, and a voice for recognizing the voice signal corrected for the feature from the unit 49 and outputting a recognition result. It comprises a recognition means 50.
The speech recognition apparatus using the microphone array of the present invention is composed of the following five element technologies as shown in FIG.
1. 1. Estimation of the position of a sound source at a short distance from the microphone array 2. Estimation of the direction of sound wave arrival of a sound source at a long distance from the microphone array. User utterance detection 4. Sound source separation processing Speech recognition processing (Japanese Patent Application 2003-320183)
Details of these elemental technologies will be described below.
(Sound source position estimation)
FIG. 5 is a functional explanatory diagram of the microphone array of the present invention.
The microphones 1, 2, 3, 4 and the microphones 5, 6, 7, 8 are arranged to face each other as shown in FIG. In addition, it is assumed that the positions of the microphones and the sound source have a relationship as shown in the figure.
A method for estimating the position of a sound source at a short distance within about 1 m from the microphone array using the microphone array will be described below.
A position of a sound source placed at an arbitrary position in the three-dimensional space is represented by a position vector P0.

An acoustic signal output from a sound source placed at an arbitrary position in the three-dimensional space is received by Q microphones arranged at an arbitrary position in the three-dimensional space. The position of each microphone is represented by a position vector _Pq .

音源から各マイクロフォンまでの伝播時間τｑは、音速をｖとすると、次式で求められる。

各マイクロフォンで受音した中心周波数ωの狭帯域信号の、音源のそれに対する利得ｇｑは、一般的に、音源とマイクロフォン間の距離Ｒｑと中心周波数ωの関数として下記のように定義される。 The distance Rq between the sound source and each microphone can be obtained by the following equation.

The propagation time τq from the sound source to each microphone can be obtained by the following equation, where the speed of sound is v.

The gain gq of the narrowband signal having the center frequency ω received by each microphone relative to that of the sound source is generally defined as a function of the distance Rq between the sound source and the microphone and the center frequency ω as follows.

例えば、利得を距離Ｒｑ（＝ｒ）だけの関数として、実験的に求めた次式のような関数ｇ（ｒ）を用いる。

中心周波数ωの狭帯域信号に関する、音源と各マイクロフォン間の伝達特性は、

For example, an experimentally obtained function g (r) as the following equation is used as a function of the gain as a function of the distance Rq (= r).

The transfer characteristics between the sound source and each microphone for the narrowband signal with the center frequency ω are:

と表される。そして、位置Ｐ０にある音源を表す位置ベクトルａ（ω，Ｐ０）を、次式のように、狭帯域信号に関する、音源と各マイクロフォン間の伝達特性を要素とする複素ベクトルとして定義する。

音源位置の推定はＭＵＳＩＣ法（相関行列を固有値分解することで信号部分空間と雑音部分空間を求め、任意の音源位置ベクトルと雑音部分空間の内積の逆数を求めることにより、音源の到来方向や位置を調べる手法）を用いて、以下の手順で行う。ｑ番目のマイクロフォン入力の短時間フーリエ変換を

で表し、これを要素として観測ベクトルを次のように定義する。

It is expressed. Then, the position vector a (ω, P0) representing the sound source at the position P0 is defined as a complex vector having a transfer characteristic between the sound source and each microphone as an element with respect to the narrowband signal, as in the following equation.

The sound source position is estimated by the MUSIC method (the signal subspace and the noise subspace are obtained by eigenvalue decomposition of the correlation matrix, and the reciprocal of the inner product of an arbitrary sound source position vector and the noise subspace is obtained. The following procedure is used. Short-time Fourier transform of qth microphone input

The observation vector is defined as follows using this as an element.

ここで、ｎはフレーム時刻のインデックスである。連続するＮ個の観測ベクトルから相関行列を次式により求める。

Here, n is an index of frame time. A correlation matrix is obtained from the continuous N observation vectors by the following equation.

とし、それぞれに対応する固有ベクトルを

とする。そして、音源数Ｓを次式により推定する。

The eigenvalues arranged in descending order of this correlation matrix

And the corresponding eigenvectors

And Then, the number S of sound sources is estimated by the following equation.

周波数帯域

および音源位置推定の探索領域Ｕを

として、 Define the matrix Rn (ω) from the noise subspace basis vectors as

frequency band

And a search area U for sound source position estimation

As

を計算する。そして、関数Ｆ（Ｐ）が極大値をとる座標ベクトルを求める。ここでは仮にＳ個の極大値を与える座標ベクトルがＰ１，Ｐ２，・・・，Ｐｓが推定されたとする。次にその各々の座標ベクトルにある音源のパワーを次式により求める。

そして、２つの閾値Ｆｔｈｒ， Ｐｔｈｒを用意し、各位置ベクトルにおけるＦ（Ｐｓ）とＰ（Ｐｓ）が次の条件を満足するときに、

Calculate Then, a coordinate vector in which the function F (P) has a maximum value is obtained. Here, it is assumed that P1, P2,..., Ps are estimated as coordinate vectors giving S local maximum values. Next, the power of the sound source at each coordinate vector is obtained by the following equation.

Then, two threshold values Fthr and Pthr are prepared, and when F (Ps) and P (Ps) in each position vector satisfy the following conditions,

連続するＮ個のフレーム時間内の座標ベクトルＰｌにおいて発声があったと判断する。
音源位置の推定処理は連続するＮ個のフレームを１つのブロックとして処理する。音源位置の推定をより安定に行うためには、フレーム数Ｎを増やす、そして／また連続するＮｂ個のブロックの全てで式（３０）の条件が満たされたら発声があったと判断する。ブロック数は任意に設定する。連続するＮフレームの時間内において、近似的に音源が静止していると見られるほどの速さで音源が移動している場合は、前記手法により音源の移動奇跡を捉えることができる。

It is determined that there is a utterance in the coordinate vector Pl within N consecutive frame times.
In the sound source position estimation process, consecutive N frames are processed as one block. In order to more stably estimate the sound source position, the number N of frames is increased, and / or it is determined that there is a utterance when the condition of Expression (30) is satisfied in all of the consecutive Nb blocks. The number of blocks is set arbitrarily. When the sound source is moving at such a speed that the sound source can be seen to be approximately stationary within the time period of consecutive N frames, the moving miracle of the sound source can be captured by the above method.

３次元空間中の音源の到来方向は２つのパラメータ（θ，φ）で表される。方向（θ，φ）から到来する音波を各マイクロフォンで受音し、そのフーリエ変換を求めることで受音信号を狭帯域信号に分解し、各受音信号の狭帯域信号毎に利得と位相を複素数として表し、それを要素として狭帯域信号毎に全受音信号分だけ並べたベクトルを音源の位置ベクトルと定義する。以下の処理において、方向（θ，φ）から到来する音波は、前述の位置ベクトルとして表現される。位置ベクトルは具体的に以下のように求められる。ｑ番目のマイクロフォンと平面ｓの間の距離ｒｑを次式により求める。 (Ambient noise direction of arrival estimation)
A method for estimating the direction in which sound waves of a sound source at a long distance from the microphone array arrive will be described below.
The plurality of microphones can be arranged at arbitrary positions in the three-dimensional space. Sound waves coming from a long distance are considered to be observed as plane waves. FIG. 5 shows, as an example, a case where a sound wave arriving from a sound source is received by eight parallel microphone arrays. In FIG. 5, a point c indicates a reference point, and the arrival direction of the sound wave is estimated around the reference point. In FIG. 5, a plane s indicates a cross section of a plane wave including the reference point c. The normal vector n of the plane s is defined as the following equation, with the direction of the vector opposite to the propagation direction of the sound wave.

The direction of arrival of the sound source in the three-dimensional space is represented by two parameters (θ, φ). Sound waves arriving from the direction (θ, φ) are received by each microphone, and the received signal is decomposed into narrowband signals by obtaining the Fourier transform, and the gain and phase are determined for each narrowband signal of each received signal. A vector that is expressed as a complex number and is arranged as an element for all received sound signals for each narrowband signal is defined as a position vector of the sound source. In the following processing, the sound wave coming from the direction (θ, φ) is expressed as the aforementioned position vector. Specifically, the position vector is obtained as follows. A distance rq between the q-th microphone and the plane s is obtained by the following equation.

距離ｒｑは平面ｓに関してマイクロフォンが音源側に位置すれば正となり、逆に音源と反対側にある場合は負の値をとる。音速をｖとするとマイクロフォンと平面ｓ間の伝播時間Ｔｑは次式で表される。

平面ｓでの振幅を基準としてそこから距離ｒｑ離れた位置の振幅に関する利得を、狭帯域信号の中心周波数ωと距離ｒｑの関数として次のように定義する。

The distance rq is positive when the microphone is located on the sound source side with respect to the plane s, and is negative when the microphone is on the opposite side of the sound source. If the speed of sound is v, the propagation time Tq between the microphone and the plane s is expressed by the following equation.

The gain related to the amplitude at a distance rq away from the amplitude in the plane s is defined as a function of the center frequency ω of the narrowband signal and the distance rq as follows.

以上より、平面ｓを基準として、各マイクロフォンで観測される狭帯域信号の利得と位相差は次式で表される。

Ｑ個のマイクで（θ、φ）方向から到来する音波を観測するとき、音源の位置ベクトルは、各マイクロフォンについて式（２６）に従い求めた値を要素とするベクトルとして次式のように定義される。 A phase difference at a position away from the phase r with respect to the phase on the plane s is expressed by the following equation.

From the above, with the plane s as a reference, the gain and phase difference of the narrowband signal observed by each microphone are expressed by the following equations.

When observing a sound wave coming from the (θ, φ) direction with Q microphones, the position vector of the sound source is defined as the following expression as a vector whose elements are values obtained according to Expression (26) for each microphone. The

音源の位置ベクトルが定義されたら、音波の到来方向推定は、ＭＵＳＩＣ法を用いて行われる。式（１５）で与えられる行列Ｒｎ（ω）を用い、到来方向推定の探索領域Ｉを

として、

を計算する。そして、関数Ｊ（θ、φ）が極大値を与える方向（θ、φ）を求める。ここでは仮にＫ個の音源が存在し、極大値を与えるＫ個の到来方向（（θ１、φ１），・・・，（θＫ、φＫ））が推定されたとする。次にその各々の到来方向にある音源のパワーを次式により求める。

When the position vector of the sound source is defined, the direction of arrival of the sound wave is estimated using the MUSIC method. Using the matrix Rn (ω) given by Equation (15), the search region I for direction of arrival estimation is

As

Calculate Then, the direction (θ, φ) in which the function J (θ, φ) gives the maximum value is obtained. Here, it is assumed that there are K sound sources, and K arrival directions ((θ1, φ1),..., (ΘK, φK)) that give maximum values are estimated. Next, the power of the sound source in each arrival direction is obtained by the following equation.

そして、２つの閾値Ｊｔｈｒ， Ｑｔｈｒを用意し、各到来方向におけるＪ（θｋ，φｋ）とＱ（θｋ，φｋ）が次の条件を満足するときに、

連続するＮ個のフレーム時間内の到来方向（θｋ，φｋ）において周囲音源があったと判断する。音波の到来方向の推定処理は連続するＮ個のフレームを１つのブロックとして処理する。到来方向の推定をより安定に行うためには、フレーム数Ｎを増やす、そして／また連続するＮｂ個のブロックの全てで式（３１）の条件が満たされたらその方向から音波の到来があったと判断する。ブロック数は任意に設定する。連続するＮフレームの時間内において、近似的に音源が静止していると見られるほどの速さで音源が移動している場合は、前記手法により音波の到来方向の移動奇跡を捉えることができる。

Then, two threshold values Jthr and Qthr are prepared, and when J (θk, φk) and Q (θk, φk) in each arrival direction satisfy the following conditions,

It is determined that there is a surrounding sound source in the direction of arrival (θk, φk) within N consecutive frame times. In the process of estimating the direction of arrival of sound waves, N consecutive frames are processed as one block. In order to estimate the direction of arrival more stably, the number of frames N is increased, and / or if the condition of equation (31) is satisfied in all the consecutive Nb blocks, the sound wave has arrived from that direction. to decide. The number of blocks is set arbitrarily. When the sound source is moving at such a speed that the sound source can be seen to be approximately stationary within the time period of consecutive N frames, the moving miracle in the direction of arrival of the sound wave can be captured by the above method. .

  The short-range sound source position estimation result and the long-distance sound source direction-of-arrival direction estimation result play an important role in the subsequent speech detection process and sound source separation process. When the power of the short-distance sound source is remarkably increased with respect to the sound wave coming from the long-distance sound source, the arrival direction estimation of the sound wave of the long-distance sound source may not be performed well. Such a case is dealt with by using the arrival direction estimation result of the sound wave of the long-distance sound source estimated immediately before the short-distance sound source is generated.

(Speech detection processing)
When there are a plurality of sound sources, it is generally difficult to specify which sound source should be recognized. On the other hand, in a system that employs an interface using voice, a user utterance region that represents a position at which a user of the system utters relative to the system can be determined in advance. In this case, even if there are a plurality of sound sources around the system by the above-described method, if the position of each sound source and the arrival direction of the sound waves can be estimated, the sound source that enters the user utterance region that the system assumes in advance is selected. Thus, the user's voice can be easily identified.
The presence of a sound source is detected when the conditions of Expression (20) and Expression (31) are satisfied, and further, the conditions of the position of the sound source and the arrival direction of the sound wave are satisfied, and the user's utterance is detected. This detection result plays an important role in the subsequent speech recognition process as the speech section information. When performing speech recognition, it is necessary to detect the start time and end time of an utterance section from an input signal. However, it is not always easy to detect an utterance section in a noise environment in which ambient noise exists. In general, if the start time of the utterance section is deviated, the speech recognition accuracy is significantly degraded. On the other hand, even if there are a plurality of sound sources, the functions represented by Expression (18) and Expression (29) show a sharp peak at the position where the sound source is and the arrival direction of the sound waves. Therefore, the speech recognition apparatus of the present invention that performs speech segment detection using this information has the advantage that it can perform speech segment detection robustly even in the presence of multiple ambient noises, and can maintain high speech recognition accuracy. Have.

For example, a user's utterance area as shown in FIG. 6 can be defined.
FIG. 6 is a functional explanatory diagram of the speech detection processing according to the present invention.
In this figure, for the sake of simplicity, only the XY plane is shown, but in general, an arbitrary user utterance region can be similarly defined in a three-dimensional space. In FIG. 6, assuming a process using eight microphones m1, m2, m3, m4, m5, m6, m7, and m8 arranged at arbitrary positions, a short-distance sound source search region and a long-distance sound source search are performed. Each of the areas defines a user utterance area. The short-distance sound source search space is a rectangular area whose diagonal is a straight line connecting two points (PxL, PyL) and (PxH, PyH). Two rectangular areas whose diagonals are straight lines connecting two points of (PTxL2, PTyL2) and (PTxH2, PTyH2) are defined as the user's utterance area. Accordingly, by selecting the sound source positions determined to have been uttered according to the equation (20) and whose coordinate vectors are within the user utterance area, the user can select among the sound sources existing at a short distance. The voice can be specified. On the other hand, the search space for the long-distance sound source defines the direction from the angle θL to θH with the point C as a reference, and defines the region from the angles θTL1 to θTH1 as the user's utterance region. Therefore, by selecting the arrival directions of the sound waves determined to have been uttered according to the equation (31) within the user utterance area, the sound sources existing at a long distance can be selected. User voice can be specified.

この相関行列の大きい順に並べた固有値を

とし、それぞれに対応する固有ベクトルを (Sound source separation processing)
A sound source separation process for emphasizing a user's voice and suppressing ambient noise using a sound source position estimation result or a sound wave arrival direction estimation result detected by speech will be described below.
The utterance position or the arrival direction of the user voice is obtained by the utterance detection process. Further, the sound source position or direction of arrival of ambient noise has already been estimated. Using these estimation results, the sound source position vectors of Equations (8) and (27), and σ representing the variance of omnidirectional noise, the matrix V (ω) is defined as follows.

The eigenvalues arranged in descending order of this correlation matrix

And the corresponding eigenvectors

とする。ここで、相関行列Ｖ（ω）には近距離音源Ｓ個と遠距離音源Ｋ個を合わせて（Ｓ＋Ｋ）個の音源が含まれているから、固有値の大きい方から（Ｓ＋Ｋ）の固有値と固有ベクトルを用いて、Ｚ（ω）を次式のように定義する。

そして、近距離の座標ベクトルＰに居るユーザの音声を強調する分離フィルタＷ（ω）は、次式で与えられる。

And Here, since the correlation matrix V (ω) includes (S + K) sound sources including S short-distance sound sources and K long-distance sound sources, the eigenvalues and eigenvectors of (S + K) in descending order of eigenvalues. Is used to define Z (ω) as follows:

A separation filter W (ω) that enhances the voice of the user in the short distance coordinate vector P is given by the following equation.

この強調されたユーザ音声の波形信号は式（３７）の逆フーリエ変換を計算することで求められる。
一方、遠距離の方向（θ，φ）に居るユーザの音声を強調する場合の分離フィルタＭ（ω）は次式で与えられる。 The voice v (ω) of the user in the coordinate vector P is obtained by multiplying the separation filter of Equation (36) by the observation vector of Equation (10).

The emphasized user speech waveform signal is obtained by calculating the inverse Fourier transform of equation (37).
On the other hand, the separation filter M (ω) for emphasizing the voice of the user in the long distance direction (θ, φ) is given by the following equation.

式（３８）の分離フィルタに式（１０）の観測ベクトルを乗じることで方向（θ，φ）に居るユーザの強調音声ｖ（ω）が得られる。

この強調されたユーザ音声の波形信号は式（３７）の逆フーリエ変換を計算することで求められる。
連続するＮフレームの時間内において、近似的に音源が静止していると見られるほどの速さで音源が移動している場合は、前記手法により移動しているユーザの強調音声が得られる。

By multiplying the separation filter of Expression (38) by the observation vector of Expression (10), the emphasized voice v (ω) of the user in the direction (θ, φ) is obtained.

The emphasized user speech waveform signal is obtained by calculating the inverse Fourier transform of equation (37).
When the sound source is moving at such a speed that the sound source can be seen to be approximately stationary within the time period of consecutive N frames, the emphasized voice of the moving user can be obtained by the above method.

(Voice recognition processing)
The sound source separation processing is effective for directional noise, but noise remains to some extent for omnidirectional noise. In addition, a noise suppression effect cannot be expected even for noise that occurs in a short time such as sudden noise. Therefore, for the recognition of the operator voice emphasized by the sound source separation process, for example, the feature correction method described in Japanese Patent Application No. 2003-320183 “Background noise distortion correction processing method and voice recognition system using the same” is used. By using a built-in speech recognition engine, the effects of residual noise are reduced. Note that the present invention is not limited to Japanese Patent Application No. 2003-320183 as a speech recognition engine, and it is conceivable to use a speech recognition engine in which various methods that are robust against noise are mounted.

  The feature correction method described in Japanese Patent Application No. 2003-320183 performs feature correction of noise superimposed speech based on a Hidden Markov Model (HMM) that a speech recognition engine has as a template model for speech recognition in advance. The HMM is learned based on Mel-Frequency Cepstrum Coefficient (MFCC) obtained from clean speech with no noise. For this reason, it is not necessary to prepare a new parameter for feature correction, and there is an advantage that the feature correction method can be incorporated into an existing recognition engine relatively easily. In this method, noise is divided into a stationary component and a non-stationary component that shows a temporary change, and the stationary component of the noise is estimated from several frames immediately before the utterance. A copy of the distribution of the HMM is generated, and the estimated noise stationary component is added to generate a feature amount distribution of the stationary noise superimposed speech. The distortion due to the stationary noise component is absorbed by evaluating the posterior probability of the observed characteristic amount of the noise superimposed speech with the feature amount distribution of the stationary noise superimposed speech. However, since distortion due to the unsteady component of noise is not taken into account only by this processing, the posterior probability obtained by the above means is not accurate when the unsteady component of noise exists. On the other hand, by using the HMM for feature correction, the temporal structure of the feature amount time series and the accumulated output probability obtained along with it can be used. By assigning the weight calculated from the accumulated output probability to the above-mentioned posterior probability, the reliability of the posterior probability deteriorated due to the non-stationary component that temporarily changes the noise can be improved.

  When performing speech recognition, it is necessary to detect the start time and end time of an utterance section from an input signal. However, it is not always easy to detect an utterance section in a noise environment in which ambient noise exists. In particular, since the speech recognition engine incorporating the feature correction estimates a steady feature of ambient noise from several frames immediately before the start of speech, the recognition accuracy is significantly deteriorated when the start time of the speech section is shifted. On the other hand, even if there are a plurality of sound sources, the functions represented by Expression (18) and Expression (29) show a sharp peak at the position where the sound source is and the arrival direction of the sound waves. Therefore, the speech recognition apparatus of the present invention that performs speech segment detection using this information can robustly perform speech segment detection even when a plurality of ambient noises exist, and can maintain high speech recognition accuracy.

  Heavy equipment such as crane trucks and excavators that cannot be used due to complex operations with high noise, and by installing a microphone array on a sofa in the living room as an interface for operating various home appliances such as TVs and videos It can be used in situations where there are various other noises such as applications, and where operation by sound and voice is required in an environment where vibrations are involved.

It is the schematic of the headset type microphone array audio | voice input apparatus of this invention. It is a block diagram which shows the means to adjust the space | interval of the storage case with both ear pads of this invention. It is a block diagram of the processing circuit accommodated in the case main body of this invention. It is a block diagram of the speech recognition processing apparatus of the present invention. It is function explanatory drawing of the microphone array of this invention. It is function explanatory drawing of the speech detection process by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Headset 2R, 2L Ear case storage case 2Ra, 2La Case main body 2Rb, 2Lb Ear pad 3 Headband 4R, 4L Post 5 Microphone 6 Microphone array

Claims (9)

A headband, a storage case with ear pads provided at both ends of the headband, and a support provided in each of the storage cases with both earpads, the support provided with any number of one or more microphones, A headset type microphone array audio input device configured to output an output signal of a microphone to the housing case with the ear pad. 2. The headset type microphone array voice input device according to claim 1, wherein a plurality of microphones are provided for each of the columns, and the microphones are arranged at a predetermined distance. 3. The headset type microphone array audio input device according to claim 1, wherein a microphone for each of the columns is provided on a surface facing both columns. 4. The storage case with each ear pad includes a case main body and an ear pad, and the case main body is provided with a circuit for outputting an audio signal obtained by performing arithmetic processing on an output signal of each microphone. A headset type microphone array voice input device according to claim 1. 5. The headset type microphone array audio input device according to claim 4, wherein an earphone for inputting the audio signal is provided in each case body, and a speaker is provided in each ear pad. 6. The headset type microphone array voice input device according to claim 4, wherein a wireless communication means is provided in each case body. The headset type microphone array audio input device according to claim 4, wherein the support column is provided in each case main body so as to freely adjust a mounting angle. The headset type microphone array audio input device according to claim 1, wherein both the support columns are provided in parallel to each other. 9. The headband has a structure in which the length of the headband can be changed by sliding relative to each other, and each case body is provided to the headband at any mounting angle. The headset type microphone array voice input device according to claim 1.

Images