JP5988077B2 - Utterance section detection apparatus and computer program for detecting an utterance section - Google Patents

Description

  The present invention relates to an apparatus and a computer program for detecting an utterance section of a specific speaker in speech recognition and the like, and in particular, in an environment where there are many utterances of another person such as a mobile phone, The present invention relates to an apparatus and a program for detecting with high accuracy.

  As computer and communication technologies have developed, the situation of being used in every scene of people's lives is emerging. In particular, with the spread of so-called smartphones, which can be called portable computers, many people have the opportunity to communicate with other terminals everywhere.

  What is a problem in this situation is the so-called user interface. In particular, when it is necessary to input a large amount of text or to input a specific character string in order to perform a specific operation on the smartphone, how to input them efficiently becomes a problem. In smartphones, it is common to display a so-called soft keyboard on a touch panel and use it to input a character string. However, due to the restriction that the smartphone must be portable, this keyboard is small, difficult to use, and because it uses a touch panel, there is a possibility that an input that is different from the intended one may be made by touching the touch panel surface slightly. . Therefore, calmness and patience are required to input a character string on a smartphone.

  One means for solving these problems is “voice” input, which is used daily by most people. If you can use your voice to give the correct input to your smartphone, you don't have to worry about a small keyboard, and you don't have to be angry about the inefficiency of typing. Usability, which is a weak point of smartphones, will improve, and there will be more opportunities to use smartphones in a wider range of life. In reality, there are examples in which a technique for recognizing a voice, understanding its contents, and responding appropriately to a question by voice is employed in a user interface of a smartphone.

  Particularly problematic in smartphones is the fact that the environment in which they are used varies and the ambient noise (environmental sound) is not constant. If it is an office, there is almost no environmental sound. Therefore, if voice recognition is performed on voice recorded using an office computer, a considerably high accuracy can be obtained. However, mobile phones are often used outdoors, and such a favorable environment cannot be expected. Particularly problematic is the detection of utterance sections in environmental sounds. Even if there is no utterance, if you try to recognize the contents as environmental sound, you can get only meaningless output. On the other hand, if there is an utterance but it is not recognized, important information may be missing from the recognition result. Therefore, in speech recognition, it is important to detect the utterance section with high accuracy.

  There are various types of environmental sounds that hinder the detection of the utterance interval, and the method of dealing with them varies depending on the type. For example, an air conditioner, an automobile engine sound, etc. are stationary noises. For such noise, noise suppression using a spectrum subtraction method, a Wiener filter, or the like is effective. In the case of non-stationary noise such as the sound of a train entering a station or the sound of a construction site, noise tracking using a particle filter is effective. On the other hand, in the case of a voice of a person other than the speaker, for example, a voice of a person in the next or back seat, it is difficult to cope with a noise suppression method that emphasizes the voice of the person. One method is a microphone array. However, since a plurality of microphones are required, it is not suitable for daily use. Therefore, it is desired that even with a single microphone, it is effective to accurately eliminate only the target speaker's utterance section while effectively eliminating background noise that is composed of human voice.

  One of the methods for detecting an utterance section uses a probability model disclosed in Non-Patent Document 1 described later. Referring to FIG. 1, one conventional technique for detecting an utterance section uses a hidden Markov model (HMM) 30. This HMM 30 has four states 44, 46, 48 and 50 arranged between a start point 40 and an end point 42. States 44, 48 and 50 correspond to states without speech (hereinafter referred to as “SIL”). The state 46 corresponds to a state where there is an utterance (hereinafter referred to as “SP”). The output probability of the acoustic parameter from the states 44, 48 and 50 is represented by a noise GMM (Gaussian Mixture Model) which is an acoustic model when there is no utterance. The output probability of the acoustic parameter from the state 46 is represented by a speech model SP that is an acoustic model prepared in advance based on the utterance. In this example, the same probability is assigned to the transition link from the state where there is a transition link as illustrated between the states to the next state. For example, there are three links exiting from the state 46 including the link to itself, and all of them are assigned a transition probability of 1/3.

  Normally, when inputting by voice recognition, the user instructs the voice recognition device to start voice recognition in some form (for example, presses a button for starting voice) and starts speaking. When the user finishes the utterance, the user instructs the voice recognition device to end the voice recognition (for example, presses a button indicating the end of the utterance). Therefore, it is assumed that there is a silent state at the beginning and the end of the speech section detection, and a time zone sandwiched between the silent states is the speech section. Furthermore, it is considered that there are silent periods in the utterance section. A model of such a transition is shown in FIG.

  Conventionally, this HMM 30 is used to calculate the probability of utterance using a speech model based on the feature amount of input speech data. Similarly, the probability that there is no utterance is calculated using a noise model. Both are compared, and when the probability obtained from the speech model is higher than the probability obtained from the noise model, it is determined that the speech is being performed.

Lee Akinobu, The Julius Book Chapter 5 Voice Segment Detection / Input Rejection, [online], [Search February 25, 2011], Internet <URL: http://julius.sourceforge.jp/juliusbook/en/ desc_vad.html>

  By using the HMM, it is possible to handle the detection of the utterance state regarding various speakers in a certain framework under various environments. However, even when the HMM is used, there is still a problem that the detection accuracy of the utterance section is low when the background includes noise including the voice of a person other than the speaker. This is due to the fact that the voice other than the speaker existing in the background is erroneously detected as the voice of the speaker. If the speech that is input to the speech recognition system contains a speech segment that is different from the target speech, a word insertion error will occur due to speech other than that of the speaker, resulting in degraded speech recognition performance. There is a problem of doing.

  Therefore, an object of the present invention is to provide an utterance section detection device that can robustly detect the utterance section of a specific speaker even in an environment where a human voice enters the background.

  An utterance section detecting device according to a first aspect of the present invention is an utterance section detecting device for detecting an utterance section of a voice signal of a specific speaker. This device uses a first statistical acoustic model that has been learned using an acoustic feature obtained from a speech signal of a specific speaker as a sound source, and an acoustic feature obtained from a speech signal for learning of an unspecified speaker as a sound source. A second statistical acoustic model that has been learned using and a third statistical acoustic model that has been learned using an acoustic feature obtained as a sound source of a learning speech signal in a state without speech It comprises an acoustic model storage means, an acoustic feature quantity calculation means for calculating and outputting an acoustic feature quantity for each frame, and a series of acoustic feature quantities output by the acoustic feature quantity computation means. For each frame to calculate the likelihood obtained from the speech signal from which the first, second and third statistical acoustic models are based, using the first, second and third statistical acoustic models. The likelihood calculation means and the likelihood calculation means Acoustic features of each frame based on the likelihood and a speech period estimation means for estimating a section obtained from the audio signal of the specific speaker.

  Preferably, the utterance section estimation unit estimates a section in which the acoustic feature amount of each frame is obtained from the voice signal of the specific speaker by state transition using the HMM using the likelihood calculated by the likelihood calculation unit. The state estimation means by HMM is included. The HMM includes first to sixth states arranged between the start point and the end point. The output probability of the acoustic feature quantity in the first, fourth, and sixth states is calculated by the likelihood calculating means using the third statistical acoustic model. The output probabilities of the acoustic feature quantities in the second and fifth states are calculated by the likelihood calculating means using the second statistical acoustic model. The output probability of the acoustic feature quantity in the third state is calculated by the likelihood calculating means using the first statistical acoustic model. The HMM further includes a link that is defined for each of the first to sixth states, a link that transitions to the self, a link that transitions from the start point to the first state and the second state, and the first state and the second state. Between the first state, the second state, the third state, the third state, and the fourth state. A transition link, a link transitioning from the third state to the fifth and sixth states, a link transitioning between the fifth state and the sixth state, a fifth state and a fifth state, respectively. 6 links from the state 6 to the end point.

  More preferably, the HMM further includes a seventh state. The output probability of the acoustic feature quantity in the seventh state is calculated by the likelihood calculating means using the second statistical acoustic model. The HMM further includes a link that transits from the seventh state to the seventh state, and a link that transits between the third state and the seventh state.

  More preferably, the transition probability assigned to each link of the HMM is determined to be the same for each state in all links starting from the state.

  The utterance interval estimation means compares the likelihoods calculated by the first, second, and third statistical acoustic models for each frame by the likelihood calculation means, and corresponds to the statistical acoustic model that gives the maximum likelihood. Sound source candidate estimation means for estimating a sound source as a sound source candidate of the frame, smoothing means for smoothing a time series of sound source candidates estimated for each frame by the sound source candidate estimation means, and smoothing by the smoothing means Means for identifying a frame sequence estimated to be obtained from a sound source corresponding to the first statistical acoustic model in the time series of the sound source candidates, as a speech section of a specific speaker; May be included.

  Preferably, the smoothing means includes means for smoothing a time series of sound source candidates estimated for each frame by the sound source candidate estimating means by a hangover method.

  The computer program according to the second aspect of the present invention causes a computer to function as each means of any of the utterance section detection devices described above.

  As described above, according to the present invention, the first statistical acoustic model that has been learned using the acoustic feature obtained from the sound signal of the specific speaker as the sound source, and the learning sound signal of the unspecified speaker as the sound source. The second statistical acoustic model learned using the obtained acoustic feature quantity and the third statistical acoustic model learned using the acoustic feature quantity obtained by using the learning speech signal without speech as a sound source The model is used to estimate whether the sound source is a specific speaker, an unspecified speaker, or a silent state. Even if it is not a silence state, a section with a high probability that it is a voice of an unspecified speaker rather than a specific speaker can be excluded from the utterance section of the specific speaker. As a result, it is possible to provide an utterance section detection device that can robustly detect the utterance section of a specific speaker even in an environment where a human voice enters the background.

It is a figure which shows the topology of HMM30 for the conventional speech area detection. It is a block diagram which shows the structure of the speech recognition system using the utterance area detection apparatus which concerns on the 1st Embodiment of this invention. It is a functional block diagram of the part relevant to utterance area detection among the smart phones which perform utterance area detection in the 1st Embodiment of this invention. It is a figure which shows the topology of HMM for the speech section detection employ | adopted with the speech section detection apparatus concerning the carrying of the 1st implementation of this invention. It is a graph which shows the precision of the speech section detection by the speech section detection apparatus which concerns on the 1st Embodiment of this invention compared with the precision by the conventional apparatus. It is a figure which compares the utterance area detection result (B) by the utterance area detection apparatus which concerns on the 1st Embodiment of this invention, and compares with the result (A) by the conventional utterance area detection apparatus. It is a functional block diagram which shows the part relevant to speech area detection among the mobile telephones using the speech area detection apparatus which concerns on the 2nd Embodiment of this invention. It is a schematic diagram which shows the speaker detection process for demonstrating operation | movement of the 2nd Embodiment of this invention. It is a schematic diagram which shows the result of the smoothing of speaker detection for demonstrating the operation | movement of the 2nd Embodiment of this invention. It is a flowchart which shows the control structure of the program for implement | achieving the process which smoothes the result of speaker detection in the 2nd Embodiment of this invention. It is a figure which shows the topology of the Markov model used in the modification of the 1st Embodiment of this invention. It is a hardware block diagram of the mobile telephone which implement | achieves the speech area detection apparatus which concerns on the 1st and 2nd embodiment of this invention.

  In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated.

[First Embodiment]
"Constitution"
FIG. 2 schematically shows the configuration of the speech recognition system 60 according to the first exemplary embodiment of the present invention. Referring to FIG. 2, the speech recognition system 60 is connected to the Internet 62 and can communicate with the speech recognition server 64 that provides speech recognition services to various terminals and the speech recognition server 64 via the Internet 62. And a mobile phone 66 that has a function for receiving a voice recognition service by the voice recognition server 64 and that employs the speech zone detecting device according to the first embodiment of the present invention.

  The mobile phone 66 is a so-called smartphone in the present embodiment, and includes a touch panel display 72 capable of touch operation, a microphone 70, and a speaker (not shown) that reproduces a telephone call.

  A frame series of speech feature values obtained as a result of speech is transmitted from the cellular phone 66 to the speech recognition server 64. A frame is a digital signal of an audio signal having a predetermined time length and a predetermined shift length. In the present embodiment, what is transmitted from the mobile phone 66 to the voice recognition server 64 is a sequence of predetermined acoustic feature values obtained from the voice of each frame. Each frame has a flag indicating whether or not the frame is an utterance section. The speech recognition server 64 provides a service for performing speech recognition for the utterance section in the transmitted speech based on the flag and transmitting the resulting text data to the mobile phone 66.

  Referring to FIG. 3, a part related to speech section detection in mobile phone 66 is a voice recognition using voice recognition server 64 for an electric signal output from microphone 70 (hereinafter referred to as “voice signal”). The processing includes a front-end processing unit 76 that performs processing on the mobile phone 66 side. The front end processing unit 76 divides the audio signal into frames having a predetermined shift length and a predetermined length, calculates a predetermined acoustic feature amount for each frame, converts the frame into a frame sequence including the acoustic feature amount, and outputs the frame sequence. At this time, the front end processing unit 76 attaches a flag indicating whether or not the frame is an utterance section to each frame.

  The cellular phone 66 further includes a transmission buffer 78 for temporarily storing each frame of the acoustic feature quantity with a flag output from the front end processing unit 76, and a frame sequence of the acoustic feature quantity stored in the transmission buffer 78 as the Internet 62. A transmission / reception unit 80 that transmits to the speech recognition server 64 via a wireless telephone line network (not shown), receives the speech recognition result from the speech recognition server 64, and passes it to the front end processing unit 76 for processing. And an application 74 that receives text data processed by the processing unit 76. The front-end processing unit 76 has a function of temporarily displaying text data received from the speech recognition server 64 on the touch panel display 72 and performing necessary editing and then passing it to the application 74. The application 74 may be any application that operates based on data input from the user.

  The front end processing unit 76 digitizes the audio signal from the microphone 70, converts the frame into a frame having a predetermined length with a predetermined shift length, and outputs each of the frames output from the frame processing unit 100. And a feature extraction unit 102 that extracts a predetermined acoustic feature amount, attaches it to each frame, and outputs it. In the present embodiment, a 12-dimensional MFCC (Mel Frequency Cepstrum Coefficient), a 12-dimensional ΔMFCC which is a time derivative of the MFCC, and a 25-dimensional feature quantity of Δ power are used as the feature quantities.

  The front-end processing unit 76 further includes a frame buffer 104 formed of a ring buffer for temporarily accumulating the frames to which the feature amount is output, which is output from the feature extraction unit 102, and a frame sequence stored in the frame buffer 104. Based on the utterance section detecting unit 112 for determining whether each frame belongs to the utterance section, and outputting each frame in order at predetermined time intervals while setting a flag on the frame determined to be the utterance section, and the utterance section Three acoustic models used by the detection unit 112 to calculate the likelihood when detecting an utterance section, that is, the unspecified speaker model 106, the specific speaker model 108, and the silence model 110, and the transmission / reception unit 80 include the speech recognition server 64. Processing unit that receives the voice recognition result (text data) received from the user and displays it on the touch panel display 72 14 and a user's input to the touch panel display 72 to edit the text data displayed on the touch panel display 72 or to hand over the edited text data to the application 74. And a control unit 116. Any acoustic model used in the present embodiment is a GMM.

  In the present embodiment, when the user executes the voice recognition process, a program for the front end process for voice recognition is launched, and the utterance is started by pressing the utterance start button displayed on the screen. When the utterance ends, the utterance end button is pressed. The utterance start button and the utterance end button are both buttons displayed on the touch panel display 72. An utterance start button is displayed when not speaking, and an utterance end button is displayed while speaking. Since it is not necessary for the front end processing unit 76 to detect a speech section when not speaking, the control unit 116 stops the operation of each component of the front end processing unit 76. When the utterance start button is pressed, the control unit 116 starts the operation of each unit of the front end processing unit 76.

  The utterance section detecting unit 112 is substantially a computer program for realizing a state transition according to the topology of the HMM when the HMM is given. In the present embodiment, an HMM 130 having a topology as shown in FIG. 4 is used as this HMM. As will be described later, the mobile phone 66 has a processor, and when the processor executes this program, the speech section detection using the HMM 130 as shown in FIG. 4 can be realized.

  Referring to FIG. 4, this HMM 130 has several nodes arranged between a start point 40 and an end point 42 and a transition link between the nodes, as shown in FIG.

  The HMM 130 specifies three SIL states 140, 146, and 148 corresponding to states that are not speech segments (silent intervals), and two SP states 142 and 150 that correspond to states being spoken by unspecified speakers, respectively. It includes an SPDx state 144 corresponding to an utterance section (hereinafter referred to as “SPDx”) by a speaker (user of the mobile phone 66) and a link between the states connecting these. In the HMM 130 shown in FIG. 4, the starting point 40 is linked to the SIL state 140 and the SP state 142. SIL state 140 is linked to SP state 142, SPDx state 144, and itself. The SP state 142 is linked to the SPDx state 144, the SIL state 140, and itself. The SPDx state 144 is linked to the SP state 150, the SIL state 148, the SIL state 146, and itself. SIL state 146 is linked to SPDx state 144 and to itself. SIL state 148 is linked to SP state 150, end point 42, and itself. SP state 150 is linked to SIL state 148, end point 42, and itself.

  In the present embodiment, links that are out of a certain state are assigned the same probability. That is, if there are three links out of a certain state, 1/3 is assigned to each link, and if it is 4, 1/4 is assigned to each link as the transition probability along that link.

The utterance section detection unit 112 shown in FIG. 3 schematically shows the configuration of a program for realizing the utterance section detection based on the HMM 130 described above. The utterance section detection unit 112 includes a constraint storage unit 120 that stores a constraint describing the topology of the HMM 130, an unspecified speaker model 106, a specific speaker model 108, and a silence for each acoustic feature amount of each frame. By applying the model 110, the likelihood that the sound of the frame is that of an unspecified speaker as a sound source, the likelihood that the sound of a specific speaker is a sound source, and silence (only environmental sound) are first to third likelihood calculator 124, 126, and 128 for calculating the likelihood of a sound source, respectively, and the constraint conditions stored in the constraint condition storing unit 120, likelihood calculating section 124,12 6 , and based on the likelihood calculated by the 12 8 calculates the state transition HMM130, determines whether or not the speech frame to be processed is uttered by a particular speaker, the frame And a model application unit 122 to set according to the judgment result the value of the beam speech segment flag output.

<Operation>
The voice recognition system 60 operates as follows. Referring to FIG. 2, the user of mobile phone 66 first activates application on mobile phone 66 for using the speech recognition service of speech recognition system 60. On the touch panel display 72, a button for instructing the start of speech is displayed. When the user presses the utterance start button, the control unit 116 shown in FIG. 3 detects the input and starts the operation of each unit of the front end processing unit 76.

  The microphone 70 converts the sound into an audio signal and gives it to the framing processor 100. This audio signal is a mixture of the user's speech, the voices of the surrounding people, and the state where there is no speech. The framing processing unit 100 digitizes this audio signal, frames it into a frame of a predetermined length with a predetermined shift time, and gives it to the feature extraction unit 102. The feature extraction unit 102 performs the above-described feature amount calculation on the digitized audio signal of each frame, and outputs a frame sequence including the feature amount. The frame buffer 104 sequentially stores and outputs this frame sequence by the FIFO method.

  The utterance section detection unit 112 uses the likelihood calculation units 124, 126, and 128 for the frames that are sequentially stored in the frame buffer 104, so that the acoustic feature amount of the frames is obtained from the likelihood obtained from the speech of an unspecified speaker. The likelihood obtained from the voice of the specific speaker that is the target of speech recognition and the likelihood obtained from the state where there is no utterance are calculated. The model application unit 122 calculates a state transition according to the HMM 130 based on the constraint conditions stored in the constraint condition storage unit 120 and these likelihoods. If it is determined as a result of the calculation that the current state is the SPDx state 144, the flag of the frame to be processed is set and the frame is output from the frame buffer 104 to the transmission buffer 78. Otherwise, the model application unit 122 resets the flag of the frame to be processed and outputs the frame from the frame buffer 104 to the transmission buffer 78.

  The transmission buffer 78 temporarily stores each frame output from the frame buffer 104. The transmission / reception unit 80 assembles a packet having a predetermined length from the frame stored in the transmission buffer 78 and transmits the packet to the voice recognition server 64.

  Referring to FIG. 2, when voice recognition server 64 receives this packet, voice recognition server 64 extracts a frame sequence from the packet and performs voice recognition with reference to a flag indicating whether or not it is a speech section. This speech recognition method may be any method as long as it uses the same acoustic feature amount as the acoustic feature amount extracted by the mobile phone 66. The voice recognition server 64 transmits the result of voice recognition (text data of the recognition result and accompanying data including recognition result candidates for each word) to the mobile phone 66.

  Referring to FIG. 3, when receiving and receiving the result of the speech recognition, the transmitting / receiving unit 80 gives the data to the text processing unit 114. The text processing unit 114 displays this text data in the speech recognition result editing area of the touch panel display 72 and allows the user to edit the text data. This editing is editing of the speech recognition result, and means, for example, a process of replacing some words in the speech recognition result with another candidate. When the editing is finished, the control unit 116 gives the text data obtained as the editing result to the application 74. The application 74 handles the text data as input to a document, for example, as in the case of input from a keyboard, or interprets it as a command and executes a designated process.

"Experimental result"
In order to confirm the effectiveness of the utterance interval detection by the method disclosed in the above embodiment, an utterance interval detection experiment was performed. As a speech database to be tested, a phoneme balance sentence database (TRA-BLA) and a travel conversation sentence database (TRA) were used for learning each GMM, and a travel conversation basic expression collection (BTEC) was used for evaluation. . All of these are available from International Telecommunications Research Institute, Inc.

  The evaluation data sets were prepared with no background noise (w / o BSN, including noise that is not speech noise) and with (w / BSN). An outline of the prepared data set is shown in Table 1.

雑音は、車及び電車等、２０種類の環境雑音の中から１５種類を選んで学習用及び適応用データベースに重畳した。残りの５種類は評価用データセット（ｗ／ｏ ＢＳＮ、ｗ／ ＢＳＮ）に重畳した。ＳＮＲは１５ｄＢ、２０ｄＢ、２５ｄＢ、及び３５ｄＢの４種類とした。ＢＳＮは発話区間の切出し対象発話とのＳＮＲが１２ｄＢになるように重畳した。

As for noise, 15 types of 20 types of environmental noise such as cars and trains were selected and superimposed on the database for learning and adaptation. The remaining five types were superimposed on the evaluation data set (w / o BSN, w / BSN). There were four types of SNR: 15 dB, 20 dB, 25 dB, and 35 dB. The BSN was superimposed so that the SNR with the utterance to be extracted in the utterance section was 12 dB.

  The acoustic feature amount was a total of 25 dimensions including 12-dimensional MFCC and ΔMFCC and Δpower. The analysis was performed at a sampling frequency of 16 kHz, a frame length of 20 milliseconds, and an analysis period of 10 milliseconds.

  For the evaluation of the utterance section detection, the False Rejection Rate (FRR) and the False Acceptance Rate (FAR) shown in the following formula were used.

ただしＮ_ｓは音声フレーム数、Ｎ_ＦＲは音声を非音声として検出したフレーム数、Ｎ_ｎｓは非音声フレーム数、Ｎ_ＦＡは非音声を音声として検出したフレーム数である。

Where N _s is the number of voice frames, N _FR is the number of frames in which voice is detected as non-voice, N _ns is the number of non-voice frames, and N _FA is the number of frames in which non-voice is detected as voice.

  Among the above experiments, the FRR and FAR obtained as a result of using the conventional technique for the speech with the background noise superimposed, and the FRR and FAR obtained as a result of using the above embodiment are shown in comparison with FIG. FIG. 5 shows the result of an experiment conducted with the number of classes C of matrix transformation set to 32 in speaker adaptation using multi-class MLLR (maximum likelihood linear regression). Referring to FIG. 5, for the test set including BSN, both the FAR and the FRR are considerably lowered by the above embodiment as compared with the conventional case. In the conventional technique, a BSN section is determined as an utterance section, whereas in the above embodiment, such an area is determined as a non-utterance section by being aligned with an unspecified speaker GMM (SP). It is.

  Give an example. Referring to FIG. 6 (A), when speech segment detection was performed on the speech signal represented by spectrogram 180 using the conventional technique, segments 192, 196, 200 and 204 were detected. Silent sections 190 and 206 were detected at the beginning and end of the audio signal, respectively, and short silent sections 194, 198, and 202 were detected in the middle of the speech section.

  On the other hand, as shown in FIG. 6B, when the speech segment detection is performed on the same speech signal using the technique of the above embodiment, the speech segment of the specific speaker is shown in FIG. Sections 242, 248, and 256, which are much shorter than the speech section detected in, were detected. As in the case of FIG. 6A, silent sections 240 and 258 are detected at the beginning and end of the audio signal, respectively, but a considerable part (sections 246, 250 and 254) were detected as speech segments by other speakers. Silent sections 244 and 252 were also detected, but these are also somewhat different from the case of FIG.

  As is clear from comparison between FIGS. 6A and 6B, it can be seen that in the prior art, the speech segment other than the speech segment of the main speaker is erroneously detected as the speech segment. Due to such erroneous detection, a word insertion error caused by speech other than the speaker occurs in the subsequent speech recognition processing. On the other hand, according to the above embodiment, it is less likely that another person's utterance is detected as an utterance section, and word insertion errors can be reduced.

  The same experiment was performed for the cases where the number of classes = 1 and 8. However, as the value of the class number C was increased, the values of FAR and FRR decreased (accuracy increased). Therefore, a certain number of classes is necessary for adaptation.

  Although the evaluation result for the data set without BSN is not shown here, the value of FRR according to the above embodiment is lower than that of the prior art, but FAR is slightly increased.

  As described above, according to the first embodiment, it is possible to detect an utterance section of a speaker with higher accuracy than in the past even in an environment where a background human voice exists. By using this result, it is possible to improve the accuracy of subsequent speech recognition. There is no need to use a plurality of microphones, and for example, it can be easily incorporated into a mobile phone or the like.

[Second Embodiment]
"Constitution"
In the first embodiment, the HMM is used for the speech section detection. This HMM has a function of smoothing a detection result when detecting an utterance section depending only on the likelihood output by each model for each frame. Similar smoothing can also be realized using other than the HMM. For example, there is a technique by hangover. In the second embodiment, the detection result of the utterance interval is smoothed not by the HMM but by the hangover method.

  Referring to FIG. 7, the mobile phone 280 according to the second embodiment is different from the mobile phone 66 according to the first embodiment in that it is described above instead of the front-end processing unit 76 of the mobile phone 66. It includes a front-end processing unit 290 that performs a process of smoothing the result of speech segment detection by the hangover method.

  The front-end processing unit 290 is connected to receive the output of the frame buffer 104, newly includes a smoothing processing unit 302 that performs smoothing processing using a ring buffer, and likelihood calculation units 124, 126, and 128. The likelihood comparison unit 300 that determines which likelihood is the highest by comparing the outputs of the output and gives the result to the smoothing processing unit 302 in place of the constraint condition storage unit 120 and the model application unit 122 is included. is there.

  The function of the smoothing processing unit 302 will be briefly described with reference to FIGS. Referring to FIG. 8, the outputs of likelihood calculating sections 124, 126, and 128 can be considered as sound source candidates in the speech section. If these are simply compared and the result is used for speech segment detection, it is determined that sound sources corresponding to the same model are sound source candidates, such as frames 320, 322, 324, 326, 328, and 330, for example. There is a case in which a series of consecutive frames is determined to be sound from a sound source corresponding to another model for a very short time (for example, one frame). Usually, since the utterance is continuously performed for a certain period of time, it is not preferable that the determination of the sound source changes intermittently in this way. Thus, for example, a frame 320 determined from a series of the same sound source as shown in FIG. 8, for example, even if a frame determined to be a sound from another sound source exists for a very short time, Processes that are considered continuous. By such processing, for example, as shown in FIG. 9, it is possible to obtain a result of speech segment detection in which sound from a certain sound source continues stably for a certain period of time.

  The smoothing processing unit 302 shown in FIG. 7 can be realized by software. For example, the method described on pages 90 to 91 of the standard ETSI ES 202 212 v1.1.2 regarding speech recognition may be adopted. FIG. 10 is a flowchart showing the control structure of a computer program that implements this standard. In this process, smoothing is performed using a ring buffer having a predetermined number (N) of storage locations. The symbols used in the following processing and their meanings are shown in the following table.

  Note that the processing shown in FIG. 10 is processing performed after the frame data is stored in the entire buffer for smoothing. In this process, the frame data is stored in the buffer and output in the FIFO manner.

Referring to FIG. 10, the program identifies the sound of the input frame using step 350 of reading the acoustic feature amount of the next frame and the acoustic model of the specific speaker, the unspecified speaker, and the silence. Step 352 for calculating likelihoods P _SPDx , P _SP , and P _SIL that are from the speaker, unspecified speaker, and silence, and the likelihood P _SPDx is greater than either of the likelihoods P _SP and P _SIL If steps 354 and 356 for determining whether or not the volume is high and the determinations of steps 354 and 356 are both positive, a flag indicating that the voice of this frame belongs to a specific speaker is set in TRUE. Step 358 and step 360 to set FALSE in other cases are included.

Furthermore this program determines the maximum length M of successive "TRUE" frame buffer, the step 362 is substituted into a variable M, the value of variable M is, the threshold value TH _P or more and the timer time L _S Step 364 for determining whether or not the value is smaller, and Step 366 for substituting the threshold value L _S for the variable T indicating the remaining time of the hangover when the determination in Step 364 is affirmative.

This program is further executed when the determination at step 364 is negative, and when the determination at step 364 is affirmative and the processing at step 366 is completed, and the value of variable M is greater than or equal to threshold value TH _L and is currently Step 368 for determining whether or not the frame number is greater than the initial transient time F _S (ie, after the initial transient time has elapsed), and the remaining hangover time when the determination in Step 368 is affirmative of including a step 370 to assign a moderate timer time L _M to a variable T, when the determination in step 368 is negative, a step 372 that assigns a short timer period L _L variable T.

  The program further determines, after steps 370 and 372, whether the value of the variable M is smaller than the threshold value THP and whether the value of the variable T indicating the hangover time is positive or not. Step 376 for subtracting 1 from the value of variable T, step 378 for determining whether or not the value of variable T is positive after the processing of steps 374 and 376, and the determination at step 378 are positive , When TRUE is output as a flag indicating whether or not the voice of the frame existing at the head of the buffer is from a specific speaker, and when the determination in step 378 is negative, FALSE Output step 382, and after steps 380 and 382, step 384 for moving the processing target to the next frame, buffer 1 shifted to the left, and a step 386 which returns control to step 350.

<Operation>
The cellular phone 280 according to this embodiment operates in the same manner as the cellular phone 66 according to the first embodiment. The only difference is that the results obtained from the three acoustic models are smoothed by the process of the hangover method shown in FIG. 10 instead of the process using the HMM according to the first embodiment.

<Modification>
In the first embodiment, the HMM 130 whose topology is shown in FIG. 4 is used. However, when the present invention is implemented in an HMM, the topology of the HMM is not limited to that shown in FIG. For example, an HMM as shown in FIG. 11 can be used.

  The HMM 400 shown in FIG. 11 is different from the HMM 130 shown in FIG. 4 in that it has a new SP state 410, a link that transitions between the SP state 410 and the SPDx state 144, and an SP that exits from the SP state 410. This is a point having a link that changes to the state 410. In other respects, the HMM 400 has the same topology as the HMM 130.

  The use of the HMM 400 has an effect that not only when there is a silent section during the utterance of a specific speaker, but also when there is noise due to utterance in the background, they can be excluded from the utterance section. Also, in this case, the topology of the model is only different from that of the model according to the first embodiment, and the software configuration for realizing the utterance section detecting device adopting the HMM 400 is the first embodiment. The software configuration may be the same.

  In the above embodiment, 25-dimensional feature values are used. However, the feature amount is not limited to this. In the above embodiment, the same probability is assigned to each link that exits from each state of the HMM. However, the present invention is not limited to such an embodiment. That is, these transition probabilities need not be equal to each other. For example, these transition probabilities may be learned from the actual environment. If such can be done, there is a possibility that the detection accuracy of the utterance section can be increased.

  In the above embodiment, all frames are transmitted from the mobile phone 66 and the mobile phone 280 to the voice recognition server 64. However, the present invention is not limited to such an embodiment. Only the frame of the speech segment may be transmitted to the voice recognition server 64. Further, in the above embodiment, only the acoustic feature amount and the flag indicating the speech section are transmitted to the speech recognition server 64, but in addition to this, the speech data itself may be transmitted to the speech recognition server 64. . In such a case, since the acoustic feature quantity can be calculated again by the voice recognition server 64, the acoustic feature quantity for detecting the utterance section of the mobile phone 66 or the mobile phone 280 and the voice recognition by the voice recognition server 64 are recognized. Therefore, it is not necessary for the feature amount to be the same. Of course, in the sense of reducing communication traffic and speeding up processing time, it is desirable to send only the feature amount and the flag of the utterance section as in the above-described embodiment.

  In the above embodiment, the speech section detection is performed by the mobile phone 66 or the mobile phone 280, and the speech recognition is performed by the speech recognition server 64. However, the present invention is not limited to such an embodiment. For example, these may all be realized by a single device (for example, a mobile phone or a computer). In that case, it is obvious that the processing for communication required in the above embodiment can be omitted.

"hardware"
FIG. 12 shows a typical hardware configuration for realizing the mobile phone 66 and the mobile phone 280 according to the first and second embodiments described above. Hereinafter, the hardware configuration relating to the mobile phone 66 will be described on behalf of these.

  12, in addition to the touch panel display 72 and the microphone 70, the mobile phone 66 has a CPU 420, a ROM 472, a cache memory (not shown), and a processor 420 having an interface with peripheral devices, and controls the processor 420. Therefore, a display control unit 440 that controls display on the touch panel display 72 and a touch sensor control unit 436 that detects a user's touch input to the touch panel display 72 and supplies the detection result to the processor 420 are included.

  The cellular phone 66 is further connected to the microphone 70 and the speaker 434, the microphone 70, the speaker 434, and the processor 420, an audio codec 430 that encodes and decodes audio, and power supply to each unit of the cellular phone 66. In addition, a power management unit 428 for monitoring the state of charge of a battery (not shown), various sensors 426 connected to the processor 420 including an acceleration sensor and a switch, and the processor 420 are used as a storage area. A memory 424 storing a program or the like for executing the processing, and a clock 422.

  The mobile phone 66 further includes a GPS (Global Positioning System) 446 connected to the processor 420, a short-range communication unit 448 that performs wireless short-range communication, and a Wi-Fi communication unit 450 that performs Wi-Fi communication. And a modem 452 for performing telephone communication by radio and a camera interface 444. A camera 442 is connected to the camera interface 444.

  The front-end processing unit 76 of the mobile phone 66 according to the first embodiment and the front-end processing unit 290 of the mobile phone 280 according to the second embodiment both have the hardware shown in FIG. Is executed by operating the hardware of the mobile phones 66 and 280 under the control of the program. The above-described memory 424 preferably includes, for example, a plurality of memory chips, and at least a part of the memory 424 is nonvolatile like a flash memory. A program for realizing the above processing is written in this nonvolatile memory, read at the time of execution, expanded into a readable / writable memory at any time, sequentially read from an address designated by a program counter (not shown), It is executed by the CPU 470. Further, the unspecified speaker model 106, the specified speaker model 108, and the silence model 110 are also written in advance in such a nonvolatile memory in the above embodiment.

  Although not described in the description of the above embodiment, it is desirable to collect the voices of a specific speaker using the mobile phones 66 and 280. Such speech can be used when learning the acoustic model of the specific speaker model 108, and can also be used for learning the unspecified speaker model 106 and the silent model 110 if background noise can also be distinguished. In this case, it is also possible to collect voices about utterances (normal calls, etc.) that the user is not aware of as the target of voice recognition, and this point collects voice data for each speaker on the voice recognition server 64 side. It is clearly advantageous compared to the case.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

30, 130, 400 Hidden Markov Model (HMM)
76, 290 Front-end processing unit 106 Unspecified speaker model 108 Specific speaker model 110 Silent model 112 Speaking section detection unit 120 Constraint storage unit 122 Model application unit 124, 126, 128 Likelihood calculation unit 300 Likelihood comparison unit

Claims (5)

An utterance interval detection device for detecting an utterance interval of a voice signal of a specific speaker,
A first statistical acoustic model that has been learned using an acoustic feature obtained from the voice signal of the specific speaker as a sound source, and an acoustic feature obtained from the voice signal for learning of an unspecified speaker as a sound source. Acoustic for storing the learned second statistical acoustic model and the learned third statistical acoustic model using the acoustic feature amount obtained by using the learning speech signal without speech as a sound source Model storage means;
An acoustic feature quantity calculating means for framing an audio signal and calculating and outputting the acoustic feature quantity for each frame;
Each frame of a series of acoustic feature amounts output by the acoustic feature amount calculating means has a likelihood obtained from a speech signal that is a source of the first, second, and third statistical acoustic models. Likelihood calculating means for calculating using the first, second and third statistical acoustic models;
Defines a state sequence consisting of a plurality of states arranged between a start point and an end point, a transition probability between states, and an output probability of an acoustic feature amount in each state for calculating likelihood by the likelihood calculating means A constraint storage means for storing a constraint representing the topology of the hidden Markov model
Based on the likelihood calculated by the likelihood calculating means, according to the constraint condition stored in the constraint condition storage means, an interval in which the acoustic feature amount of each frame is obtained from the speech signal of the specific speaker and a speech interval estimation means for estimating seen including,
The plurality of states correspond to any one of the utterance section by the specific speaker, the utterance section by the unspecified speaker, and the state without the utterance,
From the starting point of the hidden Markov model, there are only state transitions to the utterance interval by the unspecified speaker and the state without the utterance, and there is no state transition to the utterance interval by the specific speaker , utterance interval detection apparatus. The utterance interval estimation unit is configured to perform acoustic characteristics of each frame by state transition using the likelihood calculated by the likelihood calculation unit and the hidden Markov model represented by the constraint condition stored in the constraint condition storage unit. A state estimation means by a hidden Markov model for estimating an interval in which a quantity is obtained from the speech signal of the specific speaker
The Hidden Markov Model includes a first to sixth state of which is disposed between the start point and the end point,
The output probability of the acoustic feature amount in the first, fourth, and sixth states is calculated by the likelihood calculating means using the third statistical acoustic model,
The output probability of the acoustic feature quantity in the second and fifth states is calculated by the likelihood calculating means using the second statistical acoustic model,
The output probability of the acoustic feature quantity in the third state is calculated by the likelihood calculating means using the first statistical acoustic model,
The hidden Markov model further includes:
A self-transitioning link defined for each of the first to sixth states;
Links that respectively transition from the starting point to the first state and the second state;
A link that transitions between the first state and the second state;
A link that transitions from the first state and the second state to the third state, respectively;
A link that transitions between the third state and the fourth state;
Links respectively transitioning from the third state to the fifth and sixth states;
Links transitioning between the fifth state and the sixth state;
The utterance section detection device according to claim 1, comprising links that respectively transition from the fifth state and the sixth state to the end point. The Hidden Markov Models further comprises a seventh state of those the output probability of the acoustic features of the seventh state is calculated by the likelihood calculating means by using the second statistical acoustic model And
The hidden Markov model further includes:
A link that transitions from the seventh state to the seventh state;
The utterance section detection device according to claim 2, comprising a link that transitions between the third state and the seventh state. The utterance section detection device according to claim 2 or 3, wherein transition probabilities assigned to each link of the hidden Markov model are determined so as to be equal for each state in all links starting from the state. . A computer program for detecting an utterance section, which causes a computer to function as each means according to any one of claims 1 to 4 .

Images