JP2007017620A - Utterance section detecting device, and computer program and recording medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an utterance section detecting device capable of showing a constant performance level, regardless of the noise conditions. <P>SOLUTION: The utterance section detecting device 80 includes a feature value calculation part 102 for calculating two or more kinds of feature values to each frame of speech data; a feature value integrating part 106 which weights the two or more kinds of calculated feature values with respective predetermined weights to calculate an integrated score; an utterance section discriminating part 108, which performs discrimination between an utterance section and a non-utterance section for each frame of the speech data; a reference data storage part 126 and a label file creating part 122, which prepare data 124 with a label, indicating the utterance section and the non-utterance section to each frame; and an initialization control part 130 and a weighting-updating part 128, which use the labelled data 124 as learning data and learn weighting to the two or more kinds of feature values in the feature value integrating part 106 so as to optimize discrimination errors in the utterance section discriminating part 108. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a speech recognition technique, and more particularly to an utterance interval detection device for accurately detecting an utterance interval from a speech signal prior to speech recognition.

  One of the most important issues in current speech recognition technology is the realization of robust recognition in noisy environments. As methods for solving this problem, noise suppression methods such as spectrum subtraction and Wiener filter, approaches such as model adaptation to noise by MLLR (maximum likelihood linear regression) and PMC (parallel model combination) are known.

  In addition to these, speech interval detection is a very important elemental technology in speech recognition under noisy environments.

  FIG. 1 shows the concept and positioning of speech segment detection. Referring to the left side of FIG. 1, in speech recognition processing, speech segment detection processing 62 is performed on speech input 60. Then, the speech recognition process 64 is performed on the speech data included in the section determined as the speech section.

  With reference to the right side of FIG. 1, for example, it is assumed that the audio data 30 includes speech sections 42 and 46 and other sections 40, 44 and 48. In the utterance sections 42 and 46, a large waveform movement is observed. These are caused by the speaker's utterance. On the other hand, some waveforms are also seen in the sections 40, 44 and 48, but these are mainly noise data. The speech segment detection processing 62 cuts out the speech waveforms 50 and 52 included only in the speech segments 42 and 46 from the speech data 30. More specifically, in the utterance section detection process 62, the start time of the utterance section and the end time of the utterance section are determined. Then, voice recognition results 54 and 56 are obtained by performing voice recognition on the voice data existing between the start time and the end time of the speech section.

  As is clear from FIG. 1, if the utterance period is not detected correctly, the possibility that the subsequent recognition process will succeed is extremely low. This is because voice recognition is performed including the noise part. Therefore, a technique for determining the utterance interval as correctly as possible is required. In recent years, active research has been conducted on utterance section detection, and various processing methods have been proposed so far.

Patent Document 1 discloses an utterance section detection device that detects an utterance section by measuring the energy of voice data. In the technique disclosed in Patent Document 1, the threshold value of the voice energy for detecting the utterance section is changed following the change of the environmental noise data included in the voice data.
JP 2005-31632 A

  As described above, various methods for detecting an utterance section have been proposed. In particular, the technique described in Patent Document 1 is expected to enable detection of an utterance section that is robust against changes in the noise environment. However, the current situation is that there is room for further improvement in the accuracy of speech section detection, regardless of the technique described in Patent Document 1 or other techniques. In particular, many of these processing methods have a problem that the performance greatly depends on the noise condition (for example, the type of noise). Future speech recognition technology is expected to be used in various environments. Therefore, there is a need for a technique for detecting an utterance section that exhibits a certain performance under any noise condition.

  Accordingly, an object of the present invention is to provide an utterance section detecting device capable of exhibiting constant performance regardless of noise conditions.

  Another object of the present invention is to provide an utterance section detecting apparatus capable of exhibiting higher performance than the conventional technique regardless of noise conditions.

  Still another object of the present invention is to provide an utterance section detection device capable of detecting an utterance section with higher accuracy than the conventional technique regardless of noise conditions.

  An utterance interval detection device according to a first aspect of the present invention is an utterance interval detection device for detecting an utterance interval in audio data, and a plurality of types of feature amounts determined in advance for each frame of the audio data. A feature amount calculation means for calculating, and for each frame of audio data, a plurality of types of feature amounts calculated by the feature amount calculation means are respectively weighted to integrate the plurality of types of feature amounts. A feature amount integration unit for calculating a score, and an utterance interval identification unit for identifying an utterance interval and a non-utterance interval for each frame of speech data based on the integrated score calculated by the feature amount integration unit; In addition, for each frame, labeled data preparation means for preparing labeled data with a label indicating an utterance interval and a non-utterance interval, and labeled data Weight learning means for learning weights for a plurality of types of feature amounts in the feature amount integration means so that the labeled data prepared by the preparation means is used as learning data, and the identification error in the utterance section identification means satisfies a predetermined criterion Including.

  A plurality of types of feature quantities are weighted based on learning data so that identification errors satisfy a predetermined criterion, and are integrated to obtain an integrated score. The integrated score is used to identify speech / non-speech intervals in the speech data. Since multiple types of feature quantities are weighted by learning, the weights for each feature quantity are calculated appropriately according to the noise environment, and the speech and non-speech sections are identified with a constant accuracy regardless of the noise environment. Can do.

  Preferably, the plurality of types of feature amounts are based on the amplitude level of the audio signal in each frame, the number of zero crossings of the audio signal in each frame, the spectrum information of the audio signal in each frame, and the GMM log likelihood in each frame. Chosen from the group of

  Appropriate weighting by learning is performed on a plurality of types of feature values selected from these existing feature values. As a result, according to the utterance section detecting apparatus according to the present invention, it is possible to detect the utterance section with higher accuracy in most cases compared to the case where these existing feature quantities are used alone. This result was confirmed by experiments.

  More preferably, the labeled data preparation means includes an audio data acquisition means for acquiring audio data corresponding to a given reference utterance and an acoustic model for the given reference utterance in advance during operation of the utterance section detecting device. For each frame of the voice data acquired by the voice data acquisition means, the voice data acquired by the voice data acquisition means is subjected to forced alignment with the acoustic model for a given reference utterance with respect to the voice data acquired by the voice data acquisition means. On the other hand, it includes means for labeling the speech segment and the non-speech segment.

  During operation of the utterance section detection device, voice data corresponding to the reference utterance is acquired. Also, labeled data is prepared by forced alignment using an acoustic model for the reference utterance. Forced alignment with reference utterances whose contents are known in advance can be performed relatively accurately. As a result, since the learning data can be labeled accurately during the operation of the actual utterance section detecting device, accurate weighting can be calculated according to the actual noise environment.

  More preferably, the feature amount integration unit adds each of the plurality of types of feature amounts by adding a predetermined weight to each of the plurality of types of feature amounts calculated by the feature amount calculation unit for each frame of the audio data. And a weight storage means for storing a weight for a predetermined weight, and the weight learning means is labeled data prepared by the labeled data preparation means. And weight update means for updating the weight stored in the weight storage means in accordance with a predetermined correction criterion so that the identification error in the feature quantity integration means is reduced.

  The weight update means uses the labeled data prepared by the labeled data preparation means as learning data, and means for updating the weight stored in the weight storage means by the minimum classification error learning related to the identification error in the utterance section identification means. May be included.

  Experiments have shown that learning weights with minimum classification error learning can provide accuracy with a small number of reference utterances.

  When the computer program according to the second aspect of the present invention is executed by a computer, it causes the computer to operate as any of the above-described utterance section detection devices.

  A recording medium according to the third aspect of the present invention is a computer-readable recording medium on which the computer program is recorded.

  Hereinafter, the configuration and operation of an utterance section detecting apparatus according to an embodiment of the present invention and implementation by a computer will be described. The utterance section detection device according to the present embodiment assigns weights to the four types of feature amounts, detects the utterance section based on a value obtained by integrating the weighted feature amounts, and assigns weights to the feature amounts. Is characterized in that it is optimized immediately after the start of operation.

  In the following description and drawings, the same parts are denoted by the same reference numerals. Their functions and names are also the same. Therefore, detailed description thereof will not be repeated. Further, in the following description, it is assumed that the utterance section detection device forms part of the desktop voice response system. However, the utterance section detection device according to the present invention is not limited to such applications, and general speech recognition processing is performed. Needless to say, the present invention is applicable to all systems that need to detect a speech section. In this specification, the term “optimization” does not necessarily mean that the apparatus is set to the most preferable condition, but includes the case where the apparatus is set to a somewhat preferable state from the initial state.

  Furthermore, in the following embodiments, four types are used as feature amounts. However, the present invention is not limited to such an embodiment, and may be anything as long as it uses a plurality of types of feature amounts. In the following embodiments, the initial values of the weights assigned to these feature quantities are all the same value. However, the present invention is not limited to such an embodiment, and a specific value determined in advance by experiment may be set as the initial value, or the initial value may be set at random.

  Also, in the following description, two types of frame lengths, 100 msec and 25 msec, are used as units for processing audio, and are used depending on the type of feature amount. This is because if the length is such a level, it can be considered that the audio data is constant without any change in the calculation of each feature amount. Therefore, the length of the frame is not limited to these, and can be appropriately selected within a range where the processing is not hindered. Although the frame shift time is 10 msec, this shift time can also be selected as appropriate within a range that does not hinder the processing.

<Configuration>
FIG. 2 shows, in a block diagram form, the configuration of an utterance section detection device 80 according to an embodiment of the present invention. The utterance section detecting device 80 according to the present embodiment is part of a voice response system having a voice recognition device and a voice synthesis function, for example, and has parts common to other functional parts of the voice response system.

  Referring to FIG. 2, this utterance period detection device 80 samples, quantizes, digitizes, and digitizes an analog signal output from the microphone 82, and outputs from the A / D conversion processing unit 86. Utterance section detection processing section 90 for detecting the utterance section in response to the above, and a weight optimization section for performing processing for optimizing the weight used in the utterance section detection processing section 90 when the utterance section detection device 80 is turned on 92. The weight optimization unit 92 outputs voice data for generating a message that prompts the user to speak a predetermined reference utterance for the weight optimization process, and the message is sent via the voice synthesizer 94 and the speaker 84. Is output.

  The utterance section detection processing unit 90 includes a first framing processing unit 87 for framing the digital audio signal output from the A / D conversion processing unit 86 with a shift amount of 10 msec in length of 100 msec; And a second framing processor 88 for framing the audio signal output from the A / D conversion processor 86 with a shift amount of 10 msec in length of 25 msec.

  The utterance section detection processing unit 90 obtains two types of feature amounts from the frame data output from the first framing processing unit 87 and further two types of feature amounts from the frame data output from the second framing processing unit 88. A feature amount calculation unit 102 for calculating and outputting each, a first input 132 for receiving the feature amount of the sample data given from the weight optimization unit 92, and a second for receiving the feature amount output by the feature amount calculation unit 102 The selection unit 100 for selecting and outputting one of the first input 132 and the second input 134 according to the control signal given from the weight optimization unit 92, and the above four types The weight storage unit 104 for storing the weight assigned to each feature amount and the four types given from the selection unit 100 using the weights stored in the weight storage unit 104 Utterance interval for each frame by comparing the integrated score obtained from the feature amount integrating unit 106 for calculating the integrated score by integrating the feature amounts and the integrated score obtained from the feature amount integrating unit 106 Or a non-speech segment, and a speech segment identification unit 108 for labeling and outputting each frame.

The feature quantity calculation unit 102 calculates an amplitude level feature quantity calculation unit 140 for calculating the feature quantity f ⁽¹⁾ of the amplitude level of the speech waveform based on the amplitude level of the frame data output from the first framing processing unit 87. A zero-crossing number feature quantity calculating unit 142 for calculating a zero-crossing number feature quantity f ⁽²⁾ based on the zero-crossing number of the frame data output from the first framing processing unit 87, and a second Based on the spectrum information of the frame data output by the framing processor 88, the spectrum information feature quantity calculator 144 for calculating the spectral information feature quantity f ⁽³⁾ and the second framing processor 88 output A GMM log likelihood feature quantity calculation unit 146 for calculating a GMM log likelihood feature quantity f ⁽⁴⁾ based on the GMM log likelihood of the frame data. Hereinafter, the feature amount calculated by each unit of the feature amount calculation unit 102 will be described.

(1) Amplitude level of speech waveform The amplitude level of a speech waveform is the most basic feature used for speech section detection, and is implemented in various speech recognition systems. amplitude level E _t for t th frame is calculated by the following formula.

ここで、ｓ_nはフレーム内のサンプル信号に長さＮのハミング窓をかけた値である。

Here, s _n is a value obtained by multiplying the sample signal in the frame by a Hamming window of length N.

In the present embodiment, it is assumed that the feature amount in the noise section is known, and the ratio with the noise level is used for the amplitude level. That is, the feature quantity f _t ⁽¹⁾ calculated by the amplitude level feature quantity calculation unit 140 is as follows.

ただし、Ｅnは雑音区間での振幅レベル値である。なお、この特徴量の算出にあたっては、フレーム長１００ｍｓｅｃのデータを用いる。

However, En is an amplitude level value in a noise section. In calculating the feature amount, data having a frame length of 100 msec is used.

(2) Number of zero crossings (ZCR)
The number of zero crossings is the number of times that the signal level crosses zero within a certain time. This value becomes large in the voice section. Therefore, it is possible to detect the utterance section using this phenomenon. However, in practice, a constant bias value is set instead of zero, and the intersection within the bias range is generally not counted. Assuming that this feature quantity is _ft ⁽²⁾ , the feature quantity _ft ⁽²⁾ calculated by the zero-crossing number feature quantity calculation unit 142 is expressed as follows using the ratio to the noise interval as well as the amplitude level. The

ここでＺ_tは入力フレームのゼロ交差数、Ｚnは雑音区間でのゼロ交差数である。この特徴量の算出にあたっても、フレーム長１００ｍｓｅｃのものを用いる。

Here, Z _t is the number of zero crossings in the input frame, and Zn is the number of zero crossings in the noise interval. In calculating the feature amount, a frame length of 100 msec is used.

(3) Spectrum information In recent years, techniques for extracting features from a spectrum and using it for detecting an utterance section have been actively performed. An example spectrum of speech and noise is shown in FIG. In the example shown in FIG. 3, the voice 150 has more components distributed in the low frequency region than the noise 152. Both are almost the same in the high frequency range. Of course, the spectrum varies depending on the season depending on the type, whether it is speech or noise. In this embodiment, the frequency domain is divided into several channels, and the S / N ratio is calculated for each channel. The average value of the S / N ratio calculated in this way is used as the feature amount based on the spectrum information.

The feature value f _t ⁽³⁾ calculated by the spectrum information feature value calculation unit 144 is expressed by the following equation.

ここで、Ｂはチャネルの数を、Ｓ_btは入力フレームにおけるチャネルｂの平均パワーを、Ｎ_bは雑音区間におけるチャネルｂの平均パワーを、それぞれ表す。この特徴量の算出にはフレーム長２５ｍｓｅｃのデータを用いる。

Here, B represents the number of channels, S _bt represents the average power of channel b in the input frame, and N _b represents the average power of channel b in the noise interval. The feature amount is calculated using data having a frame length of 25 msec.

(4) GMM logarithmic likelihood Gaussian mixture distribution (GMM) is often used in recent years for utterance detection because it is easy to learn statistically. Here, the log likelihood ratio between the speech GMM and the noise GMM is used as the feature amount. The feature value f _t ⁽⁴⁾ calculated by the GMM log-likelihood feature value calculation unit 146 is expressed by the following equation.

ここでΘ_s及びΘ_nはそれぞれ音声ＧＭＭ及び雑音ＧＭＭのモデルパラメータセットである。このデータの算出にもフレーム長２５ｍｓｅｃのデータを用いる。

Here, Θ _s and Θ _n are model parameters sets of the speech GMM and noise GMM, respectively. Data with a frame length of 25 msec is also used for calculating this data.

  Referring to FIG. 2 again, the weight optimization unit 92, when optimizing the weight stored in the weight storage unit 104, prompts the speaker to perform a predetermined reference utterance, and the reference A feature obtained from a reference data storage unit 126 for storing a speech model for speech in advance and giving the message to the speech synthesizer 94, and sample speech data obtained by a user speaking according to the message described above. The sample data storage unit 120 for receiving and storing the amount from the feature amount calculation unit 102 and the feature amount obtained from the audio data of the sample stored in the sample data storage unit 120 in the reference data storage unit 126 For each frame of sample data by performing a forced alignment process with the speech model of the specified reference utterance A label file creation unit 122 for identifying a speech segment / non-speech segment and creating a label file storing a label of the speech segment / non-speech segment, and a label file created by the label file creation unit 122 are stored. A label file storage unit 124.

  The weight optimization unit 92 is further identified by the speech segment identification unit 108 based on the label file stored in the label file storage unit 124 and the score obtained by integrating the feature quantities of the sample data by the feature quantity integration unit 106. Based on the label for each frame obtained in this way, processing for recalculating the integrated weight is performed so as to minimize the difference between them (identification error of the utterance section identification unit 108), and the weight storage unit 104 A weight updating unit 128 for updating the stored weights, and each unit in the weight optimization unit 92 and the utterance interval detection in response to the power being supplied to the utterance interval detection device 80 and a reset signal being given. Processing for controlling the selection unit 100 and the weight storage unit 104 in the processing unit 90 to optimize the weights stored in the weight storage unit 104 by minimum error classification (MCE) learning And a initialization control unit 130 for performing.

-Weight optimization using MCE learning-
The above four feature quantities _ft ⁽¹⁾ , _ft ⁽²⁾ , _ft ⁽³⁾ , and _ft ⁽⁴⁾ are integrated with weights w ₁ , w ₂ , w ₃ , and w ₄ , respectively. . The integrated score F (x _t ) for the input frame x _{t at} a certain time t is expressed by the following equation.

また、重みｗ_kは以下の制約条件を満たすものとする。

The weight w _k satisfies the following constraint conditions.

なお、本実施の形態では重みの初期値は全て同じ値で０．２５とする。

In this embodiment, the initial values of the weights are all the same value and are 0.25.

In order to adapt the utterance section detection device 80 to the noise environment, the weight optimization unit 92 optimizes the weight w _k for integration using MCE learning. A generalized probabilistic descent method (GPD) is used for discriminative learning.

And classification measure false for the definition learning data x _t of the loss function is expressed as follows.

ここでｋは正解クラスであり、音声（ｓ）又は非音声（ｎ）に相当する。ｍは非正解クラスである。ｄ_k（ｘ_k）の値が負のときにはｘ_tが正しく分類されたことを示す。

Here, k is a correct class and corresponds to voice (s) or non-voice (n). m is the non-correct answer class. When the value of d _k (x _k ) is negative, x _t is correctly classified.

  Next, a sigmoid function approximating a step function of 0 or 1 is applied to the misclassification measure, and the loss is defined by the following equation.

ここで、γはシグモイド関数の傾きを表す。確率的降下法に基づいて損失関数を最小にすることが識別学習の目標となる。

Here, γ represents the slope of the sigmoid function. The goal of discriminative learning is to minimize the loss function based on the stochastic descent method.

・ Optimization of weights Weights are updated as follows. As described above, w ₁ , w ₂ , w _3, and w ₄ are weights for the feature amounts obtained from the amplitude level, the number of zero crossings, the spectrum information, and the GMM log likelihood. In this embodiment, there is a constraint that these weights must always be greater than zero. In order to ensure that this constraint is always satisfied in the process of updating by MCE learning, the weights w = {w ₁ , w ₂ , w ₃ , w ₄ } are converted into the following new sets ~ w. In addition, "-" used in the text of this specification is described immediately above the character immediately after in a formula.

〜ｗは、学習データが入力されるごとに以下のように更新される。ここで、ε_ｔは学習のステップを表し、データが入力されるたびに単調に減少していくものとする。

˜w is updated as follows each time learning data is input. Here, ε _t represents a learning step and decreases monotonously every time data is input.

式（１１）の右辺最終項は、以下のように展開される。

The last term on the right side of Equation (11) is expanded as follows.

ここで、

here,

であり、さらに式（１２）の最終因数の要素は次のように表される。

Further, the final factor element of the equation (12) is expressed as follows.

このようにして、〜ｗの学習が行なわれる。

In this way, learning of ~ w is performed.

  When learning of ~ w is completed, ~ w is inversely converted to w by the following expression.

ここで、式（１６）は制約条件（２）を満たすための正規化処理も含んでいる。

Here, the expression (16) includes a normalization process for satisfying the constraint condition (2).

・ Identification of utterance interval After optimizing the weights w ₁ , w ₂ , w ₃ , and w ₄ , the integrated score F (x _t ) is obtained, and the speech (utterance interval) is obtained using the following two identification functions Or non-speech (non-speech interval). In MCE learning, since it is necessary to prepare an identification function for each class to be identified, two identification functions are used in this way.

ここで、θは統合スコアのしきい値であり、予め定められている。ｇ_s（ｘ_t）がｇ_n（ｘ_t）より大きい場合にはｘ_tは音声フレームと判定され、そうでない場合は非音声フレームと判定される。この判定はフレームごとに行なう。

Here, θ is a threshold value of the integrated score and is determined in advance. When g _s (x _t ) is larger than g _n (x _t ), x _t is determined to be a voice frame, and otherwise, it is determined to be a non-voice frame. This determination is performed for each frame.

  FIG. 4 is a flowchart showing a program control structure when the weight optimizing unit 92 according to the present embodiment is realized by a computer program. Referring to FIG. 4, when the power is turned on, a process for initializing (clearing) each part of speech section detecting device 80 is performed in step 160. Subsequently, in step 162, the state of the utterance section detection device 80 is set to the initial state. That is, the initialization control unit 130 instructs the selection unit 100 to set a connection so as to receive the output from the sample data storage unit 120.

  In step 164, speech synthesis of reference data is performed. That is, the initialization control unit 130 instructs the speech synthesizer 94 to read out the text of a message prompting the generation of the reference utterance from the reference data storage unit 126. The voice synthesizer 94 performs voice synthesis on the text of this message and gives a voice signal to the speaker 84. The speaker 84 converts this sound signal into sound. This message is, for example, please repeat three times, "" Hello "". Is a message like this.

  In step 166, the input of the standard utterance that the user utters in response to the message is received from the microphone 82 and the A / D conversion processing unit 86, and in step 168, four types of feature quantities are calculated. This feature amount is stored in the sample data storage unit 120.

  In step 170, the feature amount of the sample data stored in the sample data storage unit 120 is forcibly aligned with the acoustic model stored in the reference data storage unit 126, and the utterance interval and non-existence in the sample data are determined. A speech segment is identified for each frame. However, at this time, a smoothing process is performed in order to make the speech segment and the non-speech segment large. In accordance with this identification result, a label file composed of labels specifying the utterance interval / non-utterance interval for each frame is created and stored in the label file storage unit 124.

  In step 172, it is determined by the likelihood of alignment whether the result of forced alignment is appropriate. When the result of forced alignment is not valid, the process returns to step 160 again, and the above processing is repeated. When the result of the forced alignment is valid, the process proceeds to step 174.

  In step 174, the weight is calculated by MCE learning. Specifically, the initialization control unit 130 controls the selection unit 100 so as to calculate the feature amount given to the first input 132, and is stored in the label file storage unit 124 according to the MCE learning described above. The label of the label file is compared with the identification result (label) of the utterance period / non-utterance period identified for each frame by the integrated score obtained from the feature amount given from the selection unit 100, and the mutual error is minimized. Thus, the weight storage unit 104, the feature amount integration unit 106, the utterance section identification unit 108, and the weight update unit 128 are controlled.

  When the weight calculation is completed, the finally obtained weight is stored again in the weight storage unit 104 in step 176. In step 178, the state of the utterance section detection device 80 is set to the normal state. That is, the selection unit 100 is set so as to select a feature amount from the second input 134. Therefore, thereafter, the utterance section detection processing unit 90 is calculated by the microphone 82, the A / D conversion processing unit 86, the first framing processing unit 87, the second framing processing unit 88, and the feature amount calculation unit 102. Based on the feature amount, the utterance section is detected in real time. At the same time, the initialization control unit 130 stops the operations of the sample data storage unit 120, the label file creation unit 122, and the weight update unit 128. That is, in the subsequent processing, the weight stored in the weight storage unit 104 is not updated.

<Operation>
The operation of the utterance section detecting device 80 whose configuration has been described above will be described below. Assume that the reference data storage unit 126 stores in advance text data of a message for prompting the utterance of the reference utterance and a voice model for performing the forced alignment with respect to the reference utterance. In actual operation, when the utterance section detection device 80 is turned on, the initialization control unit 130 shown in FIG. 2 initializes each section of the utterance section detection device 80. Furthermore, the initialization control unit 130 sets the selection unit 100 of the utterance section detection processing unit 90 to an initial state. That is, the input to the first input 132 is set to be selected.

  Subsequently, the initialization control unit 130 instructs the speech synthesizer 94 to perform speech synthesis of a message for prompting the user to speak the reference utterance stored in the reference data storage unit 126. To do. In response to this, the speech synthesizer 94 reads out the text of the message from the reference data storage unit 126, performs speech synthesis, generates a speech signal, and provides it to the speaker 84. The speaker 84 converts this sound signal into sound. The user is prompted by this voice and makes a predetermined utterance.

  This sound is converted into an analog sound signal by the microphone 82 and given to the A / D conversion processing unit 86. The A / D conversion processing unit 86 samples the audio signal, quantizes it, digitizes it, and provides it to the first framing processing unit 87 and the second framing processing unit 88.

  The first framing processing unit 87 frames the input audio data in units of 100 msec, and provides the frame to the amplitude level feature amount calculation unit 140 and the zero crossing number feature amount calculation unit 142. The second framing processing unit 88 frames the input speech in units of 25 msec, and provides it to the spectrum information feature amount calculation unit 144 and the GMM log likelihood feature amount calculation unit 146. The frame shift time is 10 msec.

  The amplitude level feature quantity calculation unit 140, the zero-crossing number feature quantity calculation unit 142, the spectrum information feature quantity calculation unit 144, and the GMM log-likelihood feature quantity calculation unit 146 each have an amplitude level feature quantity ( Power), the number of zero crossings, the spectrum information, and the GMM log-likelihood ratio are calculated and supplied to the sample data storage unit 120. The sample data storage unit 120 stores these feature amounts for each frame.

  Subsequently, the initialization control unit 130 controls the label file creation unit 122 to create a label file. That is, the label file creation unit 122 performs forced alignment on the acoustic model for initialization stored in the reference data storage unit 126 using the feature amount stored in the sample data storage unit 120. To specify an utterance start point and an utterance end point in the sample data. At this time, the smoothing process is performed so that both are not mixed as much as possible at the boundary between the utterance interval and the non-utterance interval. Such a forced alignment process can be said to be a form of speech recognition, but this process can be easily realized when the utterance content is known in advance as in the initialization in this embodiment.

  In this way, the label file creation unit 122 labels the utterance interval / non-utterance interval for each frame of the sample data. A label file is created by arranging the labels in the order of the frames. This label file is stored in the label file storage unit 124 shown in FIG.

  When the creation of the label file is completed, the initialization control unit 130 controls the weight storage unit 104, the feature amount integration unit 106, the utterance section identification unit 108, and the weight update unit 128 to optimize the weight. The learning process is executed. When the MCE learning process is completed, the obtained weight is newly stored in the weight storage unit 104. Further, the initialization control unit 130 sets the selection unit 100 to select the input of the second input 134 and stops the operations of the sample data storage unit 120, the label file creation unit 122, and the weight update unit 128.

  Thereafter, the audio signal digitized by the microphone 82 and the A / D conversion processor 86 is framed by the first framing processor 87 and the second framing processor 88, and the feature amount calculator 102. Given to. The feature amount calculation unit 102 calculates the above-described four types of feature amounts. This feature amount is then given to the feature amount integration unit 106 via the selection unit 100 instead of the sample data storage unit 120. The feature amount integration unit 106 integrates the given four types of feature amounts using the weights stored in the weight storage unit 104, and provides the obtained integrated score to the utterance section identification unit 108. The utterance section identification unit 108 compares the integrated score with a threshold value, determines a frame indicating an integrated score equal to or higher than the threshold value as an utterance section, and determines the other as a non-utterance section. Is given to a voice recognition device (not shown). At this time, smoothing is performed on the determination result of the utterance section.

  As described above, the utterance interval detection device 80 according to the present embodiment uses four types of feature amounts and identifies the utterance interval / non-utterance interval based on the integrated score obtained by integrating them. The weight at the time of integration is optimized using the reference data when the utterance section detecting device 80 is turned on. By this optimization, the weight value corresponding to the noise environment is determined. Therefore, according to the type of the noise environment, the weights for the four types of feature amounts are appropriately adjusted so that the speech segment / non-speech segment can be accurately identified. As a result, a constant effect can always be obtained regardless of the type of noise. Moreover, as can be seen from the experimental results shown later, the best accuracy can be obtained under almost all conditions when compared with an apparatus that uses four types of feature quantities independently. Therefore, regardless of the type of noise, it is possible to show higher performance than the conventional technology.

<Experiment and evaluation>
-Tasks and experimental conditions-
In order to evaluate the effectiveness of the utterance section detection device 80 according to the present embodiment, an utterance detection experiment in a noisy environment was performed. The voice data used was recorded from the speech of 10 speakers in a soundproof room (16 kHz, 16 bits). The number of utterances per person is 10 times, and each utterance is about 1 to 3 seconds. A pause of about 3 seconds is inserted between each utterance. As noise, three types of sensor room, machine tool, and spoken voice are prepared, and test data is created by superimposing these on voice data. The noise in the sensor room is relatively quiet and the sound of the air conditioner can be heard. Machine tool noise has a relatively high frequency component that cuts objects. Speaking noise includes the voice of a person speaking in the background, and has many components in the frequency band that overlap with the utterance that is the target of utterance detection.

  Three types of data were created with S / N ratios of 10 db and 15 db at the time of superposition for each noise. Therefore, a total of 600 test data samples (3 noise × 2 S / N ratio × 10 persons × 10 utterances) are generated. The data used for weight learning is another 10 utterances by the same speaker as the test data.

  In the present embodiment, it is necessary to calculate noise feature amounts in the equations (2), (3), and (4). In this experiment, the test data were calculated using the first 1 second of the test data that did not contain speech. The label file for learning was created manually instead of forced alignment.

  Next, features used for detecting the utterance section will be described. The frame length was 100 msec for the amplitude level and the number of zero crossings, and 25 msec for the GMM log likelihood and spectrum information. The frame period is 10 msec for each feature. The number of division channels of spectrum information is 20. The GMM uses a Gaussian distribution of a diagonal covariance matrix with 32 mixtures, and its input is 25 dimensions consisting of a 12-dimensional mel cepstrum, its primary difference (Δ), and Δ-power. For speech GMM learning, about 32,000 utterances by 304 people in an existing newspaper article reading corpus were used, and for noise GMM learning, three types of noise (each about 20 minutes) in the sensor room, office, and corridor were used. Here, only the sensor room is the noise used for the evaluation data for detecting the utterance section. The bias value for the number of zero crossings is 300.

  As an evaluation measure for detecting an utterance interval, frame-based false alarm rate (FAR) and false rejection rate (FRR) are used. FAR indicates the proportion of frames erroneously recognized as speech in all non-speech frames, and FRR indicates the proportion of frames erroneously recognized as non-speech in all speech frames.

-Experimental results-
The experimental results for six patterns of noise conditions are shown in FIGS. 5 is a sensor room (10 db), FIG. 6 is a sensor room (15 db), FIG. 7 is a machine tool (10 db), FIG. 8 is a machine tool (15 db), FIG. 9 is a speaking voice (10 db), and FIG. The result of 15 db) is shown.

  Each figure shows the result of detecting an utterance section using each feature alone and the result in this embodiment. Lines 200, 210, 220, 230, 240, and 250 in which “■” is plotted in the figure indicate the results of the speech segment detection according to the present embodiment. “Amplitude”, “Spectrum”, “Number of zero crossings”, and “GMM” indicate the results of speech interval detection when amplitude level, spectrum information, number of zero crossings, and GMM log likelihood are used alone, respectively. . In any figure, the horizontal axis corresponds to FAR, and the vertical axis corresponds to FRR. In FIGS. 7 and 8, the result of using the number of zero crossings is outside the range of the figure and does not appear. The plot in the figure corresponds to the threshold value of the discriminant function (speech segment detection in the utterance segment identification unit 108), and an operation curve as shown in the diagram was obtained by performing an experiment while changing the threshold value.

  First, consider a single feature. Since sensor room noise was used to create a noise GMM, the GMM log-likelihood result was expected to be the best, but in practice the zero crossing number showed the highest performance. In addition, the machine tool showed the spectrum information and the speech section detection performance with the highest GMM logarithmic likelihood in the spoken voice.

  From these results, it can be seen that the optimum feature amount varies depending on the noise environment. On the other hand, the utterance section detection device 80 according to the present embodiment showed results exceeding the single feature in all noise environments. This shows the effectiveness of the proposed method.

  Next, a so-called closed experiment was performed in which speech data used for weight learning was evaluated. The result of the experiment in the sensor room (S / N ratio: 10 db) is shown in the operation line indicated by “closed” in FIG. From the figure, it can be seen that the results of “closed” and “this embodiment” are almost the same. Similar results were obtained for other noise conditions. This indicates that the utterance section detection according to the present embodiment is robust against the fluctuation of the utterance.

  In addition, in order to confirm the effectiveness of weight adaptation, in the utterance section detection device 80 according to the present embodiment, the result of an experiment performed before the weight is optimized (that is, when all weights are equal) and the result after optimization The results were compared. At the same time, the experiment was performed by changing the number of utterances used for adaptation to 1, 5, and 10, and the change in performance accompanying the experiment was examined.

  Table 1 shows EER (Equal Error Rate) experimental results for test data in which each noise is superimposed at 10 db. EER is the value at the point where FAR and FRR are equal.

表１から分かるように、センサールームでは重み適応の前後で性能の変化がほとんど見られなかった。一方、工作機械と話し言葉とでは検出能力が改善された。全体としても雑音環境への適応の効果が見られる。適応に用いた発話数において比較すると、１発話より１０発話の方が若干性能が向上したが、大きな違いは見られなかった。これより、１回の発話でも重みの学習が十分に有効であることが判った。

As can be seen from Table 1, there was almost no change in performance before and after weight adaptation in the sensor room. On the other hand, the detection capability of machine tools and spoken language has been improved. As a whole, the effect of adaptation to the noise environment can be seen. Compared with the number of utterances used for adaptation, the performance of 10 utterances was slightly improved over that of 1 utterance, but no significant difference was observed. From this, it was found that learning of weights is sufficiently effective even for one utterance.

<Realization by computer>
Of the speech segment detection device 80 according to the present embodiment, the weight optimization unit 92 and the speech segment detection processing unit 90 can be realized by computer hardware and a computer program executed on the computer hardware. FIG. 11 shows an appearance of a computer system 330 as an example for realizing the weight optimization unit 92 and the utterance section detection processing unit 90, and FIG. 12 shows an internal configuration of the computer system 330.

  Referring to FIG. 11, this computer system 330 includes a computer 340 having an FD (flexible disk) drive 352 and a CD-ROM (compact disk read only memory) drive 350, a keyboard 346, a mouse 348, and a monitor 342. And a microphone 370 (corresponding to the microphone 82 shown in FIG. 2) and a speaker 372 (corresponding to the speaker 84 shown in FIG. 2).

  Referring to FIG. 12, in addition to FD drive 352 and CD-ROM drive 350, computer 340 includes CPU (Central Processing Unit) 356 and bus 366 connected to CPU 356, FD drive 352, and CD-ROM drive 350. A read only memory (ROM) 358 for storing a boot-up program and the like; a random access memory (RAM) 360 connected to the bus 366 for storing a program command, a system program, work data, and the like; a microphone 370 and a speaker 372 and a sound board 368 connected to the bus 366. The computer system 330 may further include a printer (not shown).

  Although not shown here, the computer 340 may further include a network adapter board that provides a connection to a local area network (LAN).

  A computer program for causing the computer system 330 to operate as the speech zone detection device 80 is stored in the CD-ROM 362 or FD 364 inserted in the CD-ROM drive 350 or FD drive 352 and further transferred to the hard disk 354. . Alternatively, the program may be transmitted to the computer 340 through a network (not shown) and stored in the hard disk 354. The program is loaded into the RAM 360 when executed. The program may be loaded directly into the RAM 360 from the CD-ROM 362, from the FD 364, or via a network.

  This program includes a plurality of instructions for causing the computer 340 to operate as the utterance section detection device 80 according to this embodiment. Some of the basic functions required to perform this operation are provided by operating system (OS) or third party programs running on the computer 340 or various toolkit modules installed on the computer 340. Therefore, this program does not necessarily include all functions necessary for realizing the system and method of this embodiment. This program includes only an instruction for executing the operation as the above-described speech segment detection device 80 by calling an appropriate function or “tool” in a controlled manner so as to obtain a desired result. Just go out. The operation of computer system 330 is well known and will not be repeated here.

  For example, the label file storage unit 124 and the reference data storage unit 126 shown in FIG. 2 are realized using the hard disk 354, and the sample data storage unit 120 and the weight storage unit 104 are realized by the RAM 360. Further, the function of the A / D conversion processing unit 86 shown in FIG. 2 is provided by the sound board 368.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a figure which shows the concept of an utterance area detection. It is a block diagram which shows the structure of the utterance area detection apparatus 80 which concerns on one embodiment of this invention. It is a graph which shows the example of the spectrum of noise and a voice. It is a flowchart which shows the control structure of the computer program for implement | achieving the optimization process of weight in the utterance area detection apparatus 80 which concerns on one embodiment of this invention. It is a figure which shows the experimental result when the noise of a sensor room is superimposed on an audio | voice with S / N ratio of 10db. It is a figure which shows the experimental result when the noise of a sensor room is superimposed on an audio | voice with S / N ratio 15db. It is a figure which shows the experimental result when the noise of a machine tool is superimposed on an audio | voice with S / N ratio 10db. It is a figure which shows the experimental result when the noise of a machine tool is superimposed on an audio | voice with S / N ratio 15db. It is a figure which shows the experimental result when the noise of speech is superimposed on the voice with an S / N ratio of 10 db. It is a figure which shows the experimental result when the noise of speech is superimposed on the voice with an S / N ratio of 15 db. It is an external view of the computer system which implement | achieves the speech area detection apparatus 80 which concerns on one embodiment of this invention. It is a block diagram of the computer shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 80 Speech area detection apparatus 82 Microphone 84 Speaker 86 A / D conversion process part 87 1st framing process part 88 2nd framing process part 90 Speaking area detection process part 92 Weight optimization part 94 Speech synthesizer 100 Selection part 102 feature amount calculation unit 104 weight storage unit 106 feature amount integration unit 108 utterance section identification unit 120 sample data storage unit 122 label file creation unit 124 label file storage unit 126 reference data storage unit 128 weight update unit 130 initialization control unit 140 amplitude Level feature amount calculation unit 142 Zero-crossing number feature amount calculation unit 144 Spectrum information feature amount calculation unit 146 GMM log likelihood feature amount calculation unit

Claims (7)

An utterance interval detection device for detecting an utterance interval in audio data,
Feature amount calculating means for calculating a plurality of predetermined feature amounts for each frame of the audio data;
Features for calculating the integrated score by weighting each of the plurality of types of feature amounts calculated by the feature amount calculation means for each frame of the audio data, and integrating the plurality of types of feature amounts. Quantity integration means;
Based on the integrated score calculated by the feature amount integration means, including an utterance interval identification means for identifying an utterance interval and a non-utterance interval for each frame of the voice data,
Furthermore, for each frame, labeled data preparation means for preparing labeled data with a label indicating an utterance interval and a non-utterance interval;
The labeled data prepared by the labeled data preparation means is used as learning data, and the feature quantity integration means weights the plurality of types of feature quantities so that the identification error in the utterance section identification means satisfies a predetermined criterion. An utterance section detecting device including weight learning means for learning The plurality of types of feature amounts are a group consisting of the amplitude level of the audio signal in each frame, the number of zero crossings of the audio signal in each frame, the spectrum information of the audio signal in each frame, and the GMM log likelihood in each frame. The utterance section detection device according to claim 1, which is selected from The labeled data preparation means includes:
Voice data acquisition means for acquiring voice data corresponding to a given reference utterance during operation of the utterance section detection device;
Means for pre-preparing an acoustic model for the given reference utterance;
For each frame of the speech data acquired by the speech data acquisition means, by performing forced alignment of the speech data acquired by the speech data acquisition means with the acoustic model for the given reference utterance, The utterance section detection apparatus according to claim 1, further comprising: means for labeling the utterance and the non-speech section. The feature amount integration unit adds each of the plurality of types of feature amounts by adding a predetermined weight to each of the plurality of types of feature amounts calculated by the feature amount calculation unit for each frame of the audio data. And means for calculating the integrated score;
Weight storage means for storing a weight for the predetermined weight,
The weight learning means uses the labeled data prepared by the labeled data preparation means as learning data, and stores it in the weight storage means in accordance with a predetermined correction criterion so that identification errors in the feature quantity integrating means are reduced. The utterance section detection device according to claim 1, further comprising weight update means for updating the weights that have been set. The weight update means uses the labeled data prepared by the labeled data preparation means as learning data, and updates the weight stored in the weight storage means by minimum classification error learning regarding identification errors in the utterance section identification means. The utterance section detection apparatus according to claim 4, comprising means for A computer program that, when executed by a computer, causes the computer to operate as the utterance section detection device according to any one of claims 1 to 5. A computer-readable recording medium on which the computer program according to claim 6 is recorded.

Images