JP2007093630A - Speech emphasizing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech emphasizing device capable of extracting only a target speech with high precision even in an environment wherein sound conditions change. <P>SOLUTION: The speech emphasizing device is equipped with a microphone array equipped with a plurality of microphones, an adaptive beam former which generates a signal by emphasizing a target speech signal, and a noise reducing device which suppresses noise of the output signal of the adaptive beam former. The speech emphasizing device uses as the adaptive beam former a robust generalized side-lobe canceller which is equipped with a fixed beam former, an adaptive blocking matrix, and an adaptive disturbance canceller, and has the fixed beam former and adaptive disturbance canceller adaptively controlled according to the SNR of an input signal and also uses as the noise reducing device a single-channel noise reducing device which suppresses noise by using a Wiener filter based upon a GMM. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a speech enhancement device.

  In recent years, there has been increasing interest in integrating microphone arrays into mobile platforms such as laptops, gaming machines, personal digital assistants (PDAs), and mobile phones. Even after decades of research, in particular, there is a demand for rapidly increasing Internet telephony and voice control applications, so such on-platform microphone array technology has unexplored capabilities (references). [Refer to [1].]

  Reference [1]: M.S.Brandstein and D.B.Ward, Eds., Microphone Arrays: Signal Processing Techniques and Applications, Springer, Berlin, 2001.

Such a microphone array on the platform is very effective for noise-robust automatic speech recognition, even though it seems optional for a hands-free communication system. However, one of the major problems for such applications is the high variability in acoustic conditions such as noise type, noise level, reverberation time, mobile platform movement relative to the speaker's head.
JP 2004-112701 A Japanese Patent Laid-Open No. 11-52988

  An object of the present invention is to provide a speech enhancement device that can extract only a target speech with high accuracy even in an environment where acoustic conditions vary.

  According to the first aspect of the present invention, there is provided a microphone array having a plurality of microphones, an adaptive beamformer for generating a signal in which a target audio signal is emphasized from a plurality of microphone signals obtained by the microphone array, and an adaptive beamformer. And a noise reduction device that suppresses noise on the output signal of the input signal. The adaptive beamformer includes a fixed beamformer, an adaptive blocking matrix, and an adaptive disturbance canceller. The fixed beamformer and the adaptive disturbance canceller are used for the SNR of the input signal. A robust generalized sidelobe canceller that is adaptively controlled in response to this is used, and a single-channel noise reduction device that suppresses noise using a winner filter based on GMM is used as the noise reduction device. Features.

  According to a second aspect of the present invention, in the first aspect of the present invention, the noise reduction apparatus performs an input by performing a mel filter bank analysis for each frame on the input speech signal transmitted from the adaptive beamformer. First means for obtaining a logarithmic mel spectrum corresponding to the audio signal; second means for determining whether or not the frame number of the logarithmic mel spectrum corresponding to the input audio signal obtained by the first means is greater than or equal to a predetermined value; When the frame number of the log mel spectrum corresponding to the input voice signal obtained by the first means is less than the predetermined value, the log mel spectrum obtained by the first means is based on the log mel spectrum corresponding to the input voice signal. A third means for performing a process for estimating a logarithmic mel spectrum corresponding to noise and then shifting to the next frame processing by the first means; When the frame number of the log mel spectrum corresponding to the input speech signal obtained by the means is greater than or equal to a predetermined value, the log mel spectrum corresponding to the noise obtained by the third means and the GMM are used. After designing a Wiener filter for each element distribution of the GMM, a fourth means for weighted averaging the obtained plurality of winner filters, and a weighted averaged Wiener filter obtained by the fourth means are converted into impulse responses, After obtaining the estimated clean voice signal by convolving the obtained impulse response with the input voice signal, the fifth means for shifting to the next frame processing by the first means is provided.

  According to the present invention, only the target voice can be extracted with high accuracy even in an environment where the acoustic conditions fluctuate.

  Embodiments of the present invention will be described below with reference to the drawings.

[1] Microphone array

  FIG. 1 shows a state in which a microphone array unit is attached to a personal digital assistant (PDA).

  In FIG. 1, 1 is a microphone array unit, and 2 is a portable information terminal (small PC). Reference numeral 21 denotes a display unit provided on the front surface of the portable information terminal 2. In this specification, the direction from the back side to the front side of the portable information terminal 2 is referred to as the front.

  The microphone array unit 1 includes a base portion 100 on a rectangular plate, a horizontally long first microphone holding portion 101 provided in a forward projecting shape at the upper end portion of the base portion 100, and a forward projecting shape in a right end portion of the base portion 100. A vertically long second microphone holding unit 102 is provided.

  The microphone array unit 1 is attached to the portable terminal 2 in a state where the portable information terminal 2 is placed on the base 100 of the microphone array unit 1. In a state where the microphone array unit 1 is attached to the portable information terminal 2, the first microphone holding unit 101 is disposed along the upper end surface of the portable information terminal 2, and the second microphone is held along the right side surface of the portable information terminal 2. Part 102 is arranged. The microphone array unit 1 and the portable information terminal 2 are connected by USB.

  FIG. 2 shows the arrangement of microphones in the microphone array unit 1.

  In this example, the microphone array unit 1 is provided with eight microphones M1 to M8. The microphone M5 is provided at a connection portion between the first microphone holding unit 101 and the second microphone holding unit 102. The microphones M1 to M4 are provided in the first microphone holding unit 101, and the microphones M6 to M8 are provided in the second microphone holding unit 102.

  The microphones M1 to M5 are arranged side by side in the horizontal direction with an equal interval D1. The microphones M5 to M8 are arranged in the vertical direction with an equal interval D2 and facing forward. The distance D1 is set to 2 cm in this example, and the distance D2 is set to 4 cm in this example. As each of the microphones M1 to M8, an omnidirectional condenser microphone is used.

[2] Voice recognition system

  FIG. 3 shows the configuration of the voice recognition system.

  The voice recognition system includes a personal digital assistant (PDA) 2 to which a microphone array unit 1 is attached, and a voice recognition device 3 connected to the personal digital assistant 2 through a wireless LAN.

  The microphone array unit 1 includes a multi-channel A / D converter 11 for converting audio signals received by the microphones M1 to M8 into digital signals. Multichannel audio signals x1 (t) to x8 (t) obtained by the multichannel A / D converter 11 in the microphone array unit 1 are transmitted to the voice recognition device 3 via the portable information terminal 2 by wireless LAN. Sent.

  The speech recognition device 3 includes an adaptive beamformer 31, a single channel noise reduction device 32, and a speech recognition unit 33.

  As the adaptive beamformer 31, a robust generalized sidelobe canceller (RGSC) using an adaptive blocking matrix (see literature [4], patent document 2) (see literatures [2] and [3]). ) Is used. This GSC is designed to obtain high robustness against signal deletion due to speaker movement and reverberation while highly suppressing noise.

Reference [2]: W. Herbordt, H. Buchner, S. Nakamura, and W. Kellermann, "Application of a double-talk resilient DFT-domain adaptive filter for bin-wise stepsize controls to adaptive beamforming," Proc. IEEE- EURASIP Workshop on Nonlinear Signal and Image Processing, May 2005.
Reference [3]: W. Herbordt, H. Buchner, S. Nakamura, and W. Kellermann, "Outlier-robust DFT-domain adaptive filtering for bin-wise stepsize controls, and its application to a generalized sidelobe canceller," Proc. Int. Workshop on Acoustic, Echo, and Noise Control, Septemnber 2005.
Reference [4]: O. Hoshuyama, A. Sugiyama, and A. Hirano, "A robust adaptive beamformer for microphone arrays with a blocking matrix using constrained adaptive filters," IEEE Trans. On Signal Processing, vol. 47, no. 10 , pp. 2677-2684, October 1999.

  The single channel noise reduction device 32 is connected to the adaptive beamformer 31 and removes residual noise on the output of the adaptive beamformer 31. This noise reduction device 32 is a minimum mean square error (MMSE) of the mel logarithmic spectrum energy coefficient using a Gaussian mixture model (GMM) (see [5] and [6]). : minimum mean-squared error)) Based on an estimator.

  Reference [5]: JCSegura, A. de la Torre, MC Benitez, and AM Peinado, "Model-based compensation of the additive noise for continuous speech recognition. Experiments using AURORA II database and tasks," Proc. Eurospeech, vol. l, pp.221-224, September 2001.

  Reference [6]: M. Fujimoto and Y. Ariki, "Combination of temporal domain svd based speech enhancement and gmm based speech estimation for ASR in noise-evaluation on the AURORA2 tasks," Proc. Eurospeech, pp.1781-1784, September 2003.

  The voice signal whose noise has been reduced by the single channel noise reduction device 32 is sent to the voice recognition unit 33 for voice recognition. As the speech recognition unit 33, for example, a known speech recognition system such as the version 3.3 ATRASR large vocabulary speech recognition system developed by the present applicant is used.

  The feature of the present invention resides in that an adaptive beamformer 31 composed of a robust generalized sidelobe canceller and a single channel noise reduction device 32 that suppresses noise using a Wiener filter based on GMM are combined. Although the adaptive beamformer 31 can effectively remove noise having a relatively high frequency, noise having a low frequency remains. The single channel noise reduction device 32 can effectively remove low frequency noise. Therefore, by combining these, noise in a wide frequency band can be removed.

[3] Explanation of adaptive beamformer

  FIG. 4 shows the configuration of the adaptive beamformer 31.

  The adaptive beamformer 31 includes a fixed beamformer 41, an adaptive blocking matrix 42, an adaptive disturbance canceller 43, and an adaptive control unit 44.

[3-1] Fixed Beamformer Multichannel audio signals (signals x1 (t) to x8 (t) of the microphones M1 to M8) sent from the portable information terminal 2 are input to the fixed beamformer 41. The fixed beamformer 41 generates a signal in which the target audio signal is emphasized and signals other than the target audio signal are attenuated from the multi-channel audio signals x1 (t) to x8 (t).

  The fixed beamformer 41 estimates the target direction based on the multi-channel audio signals x1 (t) to x8 (t) sent from the portable information terminal 2, and obtains the target audio signal using the estimation result. As this direction estimation processing, for example, the method disclosed in Patent Document 1 can be used. In Patent Document 1, direction estimation is performed based on signals from three microphones arranged at the vertices of an equilateral triangle. As signals of these three microphones, for example, signals x3 (t), x5 (t), and x6 (t) of M3, M5, and M6 can be used.

  As a method of obtaining a target audio signal using the estimated target direction, for example, a delay sum beamformer method is used. That is, after adding a delay to each signal received by each microphone to make the target signal in-phase, these signals are added. The amount of delay added to each microphone is obtained based on the estimated target direction.

[3-2] Adaptive Blocking Matrix The adaptive blocking matrix 42 generates a signal in which a target speech signal is attenuated by adaptive signal processing and a signal other than the target speech signal is emphasized.

  Multi-channel audio signals x1 (t) to x8 (t) sent from the portable information terminal 2 are sent to subtracters 51 to 58, respectively. On the other hand, the output signal of the fixed beam former 41 is sent to the adaptive filters 61-68. The output signal of each adaptive filter 61-68 is sent to the corresponding subtracter 51-58. Each subtracter 51-58 subtracts the output signal of the corresponding adaptive filter 61-68 from the corresponding microphone signal x1 (t) -x8 (t). The output signals of the subtracters 51 to 58 are sent to the adaptive disturbance canceller 43 and to the corresponding adaptive filters 61 to 68 for updating the filter coefficients of the adaptive filters 61 to 68.

  In each of the adaptive filters 61 to 68, the filter coefficients are updated so that the output signal power of the corresponding subtracters 51 to 58 is minimized. As a result, the output signals of the subtracters 51 to 58 are signals in which the target signal is attenuated. As will be described later, each of the adaptive filters 61 to 68 is controlled by the adaptive control unit 44.

[3-3] Adaptive Disturbance Canceller The adaptive disturbance canceller 43 removes a component correlated with the output signal group of the adaptive blocking matrix 42 from the output signal of the fixed beamformer 41. The output signal of the adaptive blocking matrix 42 is sent to the multichannel adaptive filter 71. The output of the multichannel adaptive filter 71 (the sum of the outputs of the adaptive filters constituting the multichannel adaptive filter) is sent to the subtracter 72. An output signal of the fixed beam former 41 is sent to the subtracter 72. The subtracter 72 subtracts the output signal of the multi-channel adaptive filter 71 from the output signal of the fixed beam former 41. The output signal z (t) of the subtracter 72 is sent to the single channel noise reduction device 32 as the output signal of the adaptive beamformer 31 and is sent to the multichannel adaptive filter 71 for updating the filter coefficient of the multichannel adaptive filter 71. Sent.

  In the multi-channel adaptive filter 71, the filter coefficient is updated so that the output signal power of the subtracter 72 is minimized. As a result, the output signal of the subtracter 72 is a signal in which signals other than the target signal (interference signal and noise signal) are greatly attenuated. As will be described later, the multi-channel adaptive filter 71 is controlled by the adaptive control unit 44.

[3-4] Adaptive Control Unit As pointed out in the above document [4], the blocking matrix 42 should be adapted when the SNR (signal to noise ratio) is high. That is, when the SNR is high, the coefficients of the adaptive filters 61 to 68 in the blocking matrix 42 should be updated. On the other hand, the adaptive disturbance canceller 43 should be adapted when the SNR is low. That is, when the SNR is low, the coefficient of the multichannel adaptive filter 71 in the adaptive disturbance canceller 43 should be updated.

  Therefore, the adaptive control unit 44 adapts the blocking matrix 42 when only voice is detected. Further, the adaptive disturbance canceller 43 is adapted when only noise is detected. During so-called double talk, all adaptations are stopped. The adaptive control unit 44 constantly detects these three states, “target speech only”, “noise only”, and “double talk” in the DFT region in each frequency band.

  FIG. 5 shows the configuration of the adaptive control unit 44.

  Similarly to the fixed beamformer 41, the fixed beamformer 201 outputs a signal in which a target audio signal is emphasized and signals other than the target audio signal are attenuated from the multichannel audio signals x1 (t) to x8 (t). Generate.

  The fixed beamformer 201 enhances the target signal compared to noise so that the output signal y (t) of the fixed beamformer 201 is regarded as an estimated value of the target signal. Multi-channel audio signals x1 (t) to x8 (t) are sent to subtracters 211 to 218, respectively. The output signals y (t) of the fixed beam former 201 are sent to these subtracters 211 to 218.

  Each of the subtracters 211 to 218 subtracts the estimated value y (t) of the target signal from the corresponding audio signal x1 (t) to x8 (t), thereby estimating the noise estimated values c1 (t) to c8 (t). Is calculated.

  The output signal y (t) of the fixed beamformer 201 is converted into a DFT domain by a discrete Fourier transformer (DFT) 220, and its power spectrum density (PSD) is estimated by a PSD estimator 230. The PSD of y (t) is represented by SYY (n, t). n = 1... N, which is an index of the frequency band of DFT. t is a time index.

  The estimated values c1 (t) to c8 (t) of each noise are converted into DFT regions by corresponding DFTs 221 to 228, respectively, and their power spectrum density (PSDs) is estimated by PSD estimators 231 to 238. The power spectrum density (PSDs) of c1 (t)... c8 (t) is averaged in the adder 239. The averaged power spectrum density is represented by Scc (n, t).

  SYY (n, t) and Scc (n, t) can be regarded as an estimated value of the power spectrum density of the target speech and an estimated value of the power spectrum density of noise, respectively. The divider 241 calculates R (n, t) = SYY (n, t) / Scc (n, t). R (n, t) is an estimated value according to the frequency band of the SNR of the microphone. The maximum value of R (n, t) corresponds to a high SNR, and the minimum value of R (n, t) corresponds to a low SNR.

  In the determination unit 242, the maximum value and the minimum value of R (n, t) are detected. When the maximum value of R (n, t) is detected, the decision unit 242 determines that Db (n, t) = 1 and Di (n, t) = 0 in order to adapt only the blocking matrix 42. To do. On the other hand, when the minimum value of R (n, t) is detected, Db (n, t) = 0 and Di (n, t) = 1 are set in order to adapt only the adaptive disturbance canceller 43. In other cases, in order to stop the adaptation of the blocking matrix 42 and the adaptive disturbance canceller 43, Db (n, t) = 0 and Di (n, t) = 0.

[4] Description of single channel noise reduction device

  The single channel noise reduction device 32 suppresses noise using a Wienner filter based on GMM. A Wienner filter design method based on GMM and a noise suppression method will be described below.

[4-1] Design of Wienner filter based on GMM First, assuming that a clean speech signal is s (t) and noise is n (t), the speech signal z (t) on which noise is superimposed is expressed by the following equation (1). It is expressed by

  Next, a mel spectrum is obtained by applying discrete Fourier transform (DFT) and mel filter bank analysis to the above equation (1). The relationship between noise superimposed speech, clean speech, and noise on the mel spectrum is expressed by the following equation (2).

In the above equation (2), Z _b ^lin (i), S _b ^lin (i), and N _b ^lin (i) indicate noise superimposed speech, clean speech, and noise mel spectrum, respectively, and b is a mel filter. I indicates a frame number.

Based on the above definition, S _b ^lin (i) is estimated from Z _b ^lin (i) by the Wienner filter. Estimation by the Wienner filter is performed by the following equation (3).

The second term on the right side of the above equation (3) is the Wienner filter, and among the Wienner filter parameters, the noise mel spectrum N _b ^lin (i) is a stationary signal, and the start of the input noise superimposed speech Assuming that only a noise signal exists in 10 frames, the estimation is performed by the following equation (4).

Next, the parameter S _b ^lin (i) of the Wienner filter is a mel spectrum of clean speech, which is an unknown parameter when designing the filter. Therefore, a probability model of clean speech is created in advance using a normal distribution, and the average value μ _b ^{S, lin} of the normal distribution is substituted as the parameter S _b ^lin (i) of the Wienner filter. As a result, the above equation (3) is expressed by the following equation (5).

  The above is the formulation on the mel spectrum, but the mel spectrum is a parameter with a large amount of time change (dynamic range), and is not suitable as a modeling parameter when creating a clean speech probability model. Therefore, modeling is performed on the log mel spectrum whose dynamic range is smoothed by the logarithmic function, and the design of the Wienner filter is also performed on the log mel spectrum. Thereby, following Formula (6), (7), (8) is obtained.

  In addition, the normal distribution is used for the probability model of clean speech, but the modeling performance is low with a single normal distribution. Therefore, modeling is performed by a GMM having a plurality of normal distributions as in the following equation (9).

In the above equation (9), P _k is the weight of the element distribution k, and K is the total number of element distributions. S (i) is a vector having S _b (i) as an element, μ _k ^S is a vector having an average value μ _{b, k} ^S in an element distribution k, and Σ _k ^S is a variance value σ in the element distribution k. It is a diagonal matrix with _{b and k} ^S as elements.

  By using the GMM of the above formula (9), it is possible to design a Wienner filter for each element distribution k as shown by the following formula (10).

  Finally, a plurality of designed Wienner filters are collected as one filter by the weighted average of the following equation (11).

P _{k, i in} the above equation (11) is a weight used for the weighted average, and for this, the posterior probability of the noise superimposed speech given by the following equation (12) with respect to the noise superimposed speech GMM is used. Since the GMM that can be prepared in advance in this method is only a clean speech GMM, the parameters of the noise superimposed speech GMM are approximately generated using the following equations (13) and (14).

In the above equation (12), P _k uses the parameters of the clean speech GMM as it is. Z (i) is a vector having Z _b (i) as an element, μ _k ^Z is a vector having an average value μ _{b, k} ^Z in the element distribution k, and Σ _k ^Z is a variance value σ in the element distribution k. It is a diagonal matrix with _{b and k} ^Z as elements.

In this method, a plurality of Wienner filters are designed using GMM parameters, and a final filter is obtained by weighted averaging. Since the weights P _{k, i} are calculated for each short time frame, the filter obtained by this method is a filter that is variable in the time direction.

[4-2] Application of Wienner filter The Wienner filter designed by the above equation (11) is applied to noise superimposed speech. First, since the obtained filter is designed on the log mel spectrum, it is converted into a filter on the mel spectrum using an exponential function as shown in the following equation (15).

Next, inverse DCT is applied to ^{ _{circumflex over} (G) ^} _b ^lin (i) to convert it into an impulse response. Here, since {circumflex over (G)} _b ^lin (i) is a Wienner filter on the mel frequency, conventional inverse DCT cannot convert it into a time domain parameter (impulse response). Therefore, MEL-warped IDCT (see reference [7]) that performs conversion by mapping mel frequencies to linear frequencies is used. Thereby, the following equation (16) is obtained.

  Reference [7]: ETSI ES 202 050 V1.1.3, "Speech Processing, Transmission and Quality Aspects (STQ); Distributed Speech Recognition; Advanced Front-end Feature Extraction Algorithm; Compression Algorithms," Nov.2003.

In the above equation (16), ξ _{b, t} is a coefficient of MEL-warped IDCT.

  Finally, an estimated clean speech signal ｓs (t) is obtained by convolving the obtained impulse response with the noise superimposed speech signal z (t) as shown in the following equation (17).

[4-3] Processing Procedure of Single Channel Noise Reduction Device 32 FIG. 6 shows a processing procedure of the single channel noise reduction device 32.
First, mel filter bank analysis is performed on the noise superimposed speech signal z (t) sent from the adaptive beamformer 31 to obtain a log mel spectrum Z _b (i) (step S1).

And it is discriminate | determined whether i is 10 or more (step S2). i is less than 10 (i <10), to estimate the noise ^ N _b based on the equation (8) (step S3). Then, i is incremented by 1 (step S4), and the process returns to step S1 to shift to the next frame processing.

In step S2, if i is 10 or more (i ≧ 10), Wienner filter G _{b, k} (μ _{b, k} ^S , ^ N _b ) for each element distribution of GMM based on the above equation (10). Is designed (step S5). Next, the weight P _{k, i} is calculated based on the above equations (11) to (14), and the weighted average of the Wienner filter G _{b, k} (μ _{b, k} ^S , ^ N _b ) is performed ( Step S6). Next, based on the above equations (15) and (16), the weighted average Wienner filter ^ G _b (i) is converted into an impulse response g (t) (step S7). Then, based on the above equation (17), the impulse response g (t) is convoluted with the noise-superimposed speech z (t) to obtain an estimated clean speech signal ^ s (t) (step S8). i is incremented by 1 (step S4), and the process returns to step S1 to shift to the next frame processing.

[5] System evaluation

  As the speech recognition unit 33, a version 3.3 ATRASR large vocabulary speech recognition system developed by the present applicant was used. The feature vector consists of 12-dimensional MFCC, 12-dimensional ΔMFCC and Δlogarithmic power extracted from a 20-ms frame obtained by shifting data recorded at a 16 KHz sample rate by 10 ms. Further cepstrum average subtraction (CMS) was applied. Using a Japanese voice gender-dependent acoustic model of clean speech from speech dialogue corpus of travel arrangement work created by the applicant, speech data of 5 hours and text-to-speech that takes into account 25-hour phoneme balance I learned. Here, a phoneme HMM having 2086 states generated from the MDL-SSS algorithm (see reference [8]) was used.

  Reference [8]: T.Jitsuhiro, T.Matsui, and S.Nakamura, "Automatic generation of non-uniform HMM topologies based on the MDL criterion," IEICE Trans. On Information and Systems, vol. E87-D, no. 8, pp. 2121-2129, August 2004.

  This speech recognition system uses a multi-class composite bigram language model (see reference [9]) and a word trigram language model for rescoring. The language model is learned in a natural utterance database (SDB), a language database (LDB), and a spoken language database (SLDB) of a total of 6,300,000 words (see document [10]). The dictionary size is 55,000 words.

Reference [9]: H. Yamamoto, S. Isogai, and Y. Sagisaka, "Multi-class composite N-gram language model," Speech Communication, vol. 41, no. 2-3, pp. 369-379, October 2003.
Reference [10]: T. Takezawa, T. Morimoto, and Y. Sagisaka, "Speech and language databases for speech translation research in ATR," Proc. Int. Workshop on East-Asian Language Resources and Evaluation (EALREW), PP. 148-155, May 1998.

  To test the noise reduction system, a PDA microphone array (microphone array unit 1 in FIG. 1) and a close-talking microphone as a reference, in two acoustic environments, an ATR demonstration room and a cafeteria, A small database was recorded. Two male speakers and two female speakers read 102 utterances in each environment from the basic travel expression corpus (BTEC) test set-01 (see reference [11]).

  Reference [11]: T. Takezawa, E. Sumita, F. Sugaya, and H. Yamamoto, "Towards a broad-coverage bilingual corpus for speech translation of travel conversations in the real world," Proc. Int. Conference on Language Resources and Evaluation, vol. 1, pp. 147-152, May 2002.

All speaker locations are the same. PDA2 was placed on a lookout. The speaker was allowed to move his head when reading a reading manuscript placed next to the PDA on the platform. The reverberation times of the demonstration room and cafeteria are approximately T ₆₀ = 250 ms and T ₆₀ = 1s, respectively. Since the distance between the speaker's head and the PDA is approximately 50 cm, a high direct sound to reverberation ratio can be obtained. In the demonstration room, there were noises from several PC fans and air conditioners. In the cafeteria, there was kitchen noise, talking voice, air conditioner and refrigerator noise. Tables 1 and 2 show the average SNR of each speaker in the demonstration room and cafeteria, respectively.

  Table 1 shows the experimental results in the demonstration room, and Table 2 shows the experimental results in the cafeteria.

  In Tables 1 and 2, “F1” and “F2” represent female speakers, and “M1” and “M2” represent male speakers. The total average of all speakers was calculated in the linear region. The frequency band is in the range of 50 Hz to 8 kHz.

  In Tables 1 and 2, “Close-talk” means the average word accuracy [%] when speech recognition is performed by directly inputting the speech signal obtained by the close-talking microphone into the speech recognition unit. Is shown. “Baseline” indicates the average word accuracy [%] of all microphones when speech recognition is performed by directly inputting the speech signal obtained by the microphone to the speech recognition unit for each microphone of the microphone array. ing.

   “MMSE” means that for each microphone of the microphone array, the speech signal obtained by the microphone is input to the speech recognition unit via the MMSE (single channel noise reduction device 32) and speech recognition is performed. The average word accuracy [%] of all the microphones is shown.

   “RGSC” indicates the correct word accuracy [%] when the speech recognition unit performs speech recognition after beamforming the multi-channel speech signal obtained by the microphone array by the RGSC (adaptive beamformer 31). Yes.

   “RGSC + MMSE” means that a multi-channel audio signal obtained by a microphone array is beam-formed by RGSC (adaptive beamformer 31) and noise is reduced by MMSE (single channel noise reduction device 32), and then a voice recognition unit Indicates the correct word accuracy [%] when speech recognition is performed. The SNR is averaged in the linear region.

[5-2] Speech recognition performance

[5-2-1] Demonstration Room Table 1 shows the results of experiments in the demonstration room. The average SNR for all speakers is 24.9 dB. The average word accuracy of all speakers for the close-talking microphone (average word accuracy of “Close-talk”) is 96.74%. Since the SNR is high and the reverberation time is relatively short, the average word accuracy of “Baseline” is 93.25%. “MMSE” or “RGSC” both show a 1% improvement over “Baseline”. In “RGSC + MMSE”, the correct word accuracy is 95.57%, which is close to the performance of a close-talking microphone.

[5-2-2] Cafeteria Table 2 shows experimental results in the cafeteria. The average word accuracy of “Baseline” is 79.42%, and the noise level is high and the reverberation time is long in the cafeteria, so it is significantly lower than the result in the demonstration room. In "MMSE" or "RGSC", word accuracy is improved as in the demonstration room. “RGSC + MMSE” is close to the performance of a close-talking microphone.

[5-2-3] Variable SNR
In order to study the performance of the ASR system for variable SNR, voice data recorded in the cafeteria was used and cafeteria noise with variable levels was added. Noise was recorded separately with a PDA microphone array placed on a lookout at the same location where the voice recording was performed. The results are shown in Table 3. The value in parentheses is the standard deviation of the correct word accuracy for all microphones.

   It can be seen that “MMSE” is slightly better than “RGSC”. However, "RGSC + MMSE" complements RGSC and MMSE, and the word accuracy is improved much more than the case of each alone ("MMSE" or "RGSC") or "Baseline". It has been.

It is a perspective view which shows the state in which the microphone array unit was attached to the portable terminal. FIG. 3 is a plan view showing the arrangement of microphones in the microphone array unit 1. It is a block diagram which shows the structure of a speech recognition system. 3 is a block diagram showing a configuration of an adaptive beamformer 31. 3 is a block diagram showing a configuration of an adaptive control unit 44. FIG. 4 is a flowchart showing a processing procedure of a single channel noise reduction device 32.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Microphone array unit 2 Portable information terminal 3 Voice recognition apparatus 31 Adaptive beamformer 32 Single channel noise reduction apparatus 33 Voice recognition part M1-M8 Microphone

Claims (2)

A microphone array having a plurality of microphones, an adaptive beamformer that generates a signal in which the target audio signal is emphasized from a plurality of microphone signals obtained by the microphone array, and noise on the output signal of the adaptive beamformer are suppressed. Noise reduction device,
A robust generalized sidelobe canceller is used as the adaptive beamformer, which includes a fixed beamformer, adaptive blocking matrix, and adaptive disturbance canceller, and the adaptive beamformer and adaptive disturbance canceller are adaptively controlled according to the SNR of the input signal. ,
A speech enhancement apparatus, wherein a single-channel noise reduction apparatus that suppresses noise using a GMM-based Wiener filter is used as the noise reduction apparatus. Noise reduction device
A first means for obtaining a logarithmic mel spectrum corresponding to an input voice signal by performing a mel filter bank analysis for each frame on the input voice signal sent from the adaptive beamformer;
Second means for determining whether or not the log mel spectrum frame number corresponding to the input speech signal obtained by the first means is equal to or greater than a predetermined value;
When the frame number of the log mel spectrum corresponding to the input voice signal obtained by the first means is less than a predetermined value, the log mel spectrum obtained by the first means is based on the log mel spectrum corresponding to the input voice signal. A third means for performing a process for estimating a logarithmic mel spectrum corresponding to noise and then proceeding to a next frame process by the first means;
When the log mel spectrum frame number corresponding to the input speech signal obtained by the first means is greater than or equal to a predetermined value, the log mel spectrum corresponding to the noise obtained by the third means and the GMM are used. After designing a winner filter for each element distribution of GMM, the fourth means for weighted averaging of the obtained plurality of winner filters, and the weighted averaged winner filter obtained by the fourth means are converted into impulse responses And, after obtaining the estimated clean speech signal by convolving the obtained impulse response with the input speech signal, fifth means for shifting to the next frame processing by the first means,
The speech enhancement apparatus according to claim 1, further comprising:

Images