JP2008241970A - Speaker adaptation device, speaker adaptation method and speaker adaptation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To fully automatically perform adaptation to speaker's speech, without forcing a user selection. <P>SOLUTION: A sound analysis section 11 extracts a sound feature amount of input speech. In a sound model data base 13 and language model data base 14, a statistical sound model and a language model for performing speech recognition based on the extracted sound feature amount are stored. A search processing section 12 performs search processing by applying a statistical model on the sound feature amount, and outputs a speech recognition result. A recognition result text which is output by the search processing is corrected by the user in a recognition result correction section 15. An adaptation utilization determining section 17 calculates a reliability score for the correction text, and determines a threshold for whether or not the correction text is utilized for speaker adaptation. A speaker adaptation section 18 performs speaker adaptation, when it is determined that the correction text is utilized for speaker adaptation. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a speech recognition speaker adaptation device, a speaker adaptation method, and a speaker adaptation program for performing speech input on a computer or a portable terminal.

  In general, as shown in FIG. 3, the speech recognition apparatus is composed of two main elements, an acoustic analysis unit 101 and a search processing unit 102.

  In FIG. 3, an input voice signal is sent to the acoustic analysis unit 101, and an acoustic feature quantity is extracted from the input voice by the acoustic analysis unit 101.

  As shown in FIG. 4, the acoustic analysis unit 101 cuts out a frame having a length T from the input speech in the speech section, and extracts an n-dimensional acoustic feature amount representing the feature from the speech signal in this frame. At this time, as shown in FIG. 4, the frame cut-out position is advanced while being shifted by ΔT, and the acoustic feature amount extraction processing is executed from the beginning to the end of the speech section.

  In FIG. 3, the acoustic feature amount extracted by the acoustic analysis unit 101 is sent to the search processing unit 102. The acoustic model database 103 and the language model database 104 store statistical acoustic models and language models created in advance.

  The search processing unit 102 performs search processing according to statistical acoustic models and language models in the acoustic model database 103 and the language model database 104 based on the acoustic feature amount, and outputs a speech recognition result from the search result.

  The search processing unit 102 searches for the most probable input speech among the translatable word strings defined by the language model. As the language model, either a fixed grammar model in which a transition pattern of a word is defined in advance or a probabilistic grammar model in which a next transitionable word is stochastically determined according to a word string determined by a certain time is used. It is done.

  FIG. 5 is an example of a fixed grammar model. In the example of the fixed grammar model shown in FIG. 5, there are four words “Ito”, “Itoi”, “Imai”, “Doi” that can be transitioned from the first silent state “Sil”, and the only transition after that. A word string is defined that finally transitions to the silent state “Sil” again via the possible word “I”. That is, in this case, it is searched for the most likely word string among “[Sil] {Ito | Itoi | Imai | Doi} is [Sil]”.

  Regardless of whether the fixed grammar model or the probabilistic grammar model is used, the search processing using the acoustic feature amount for each frame is advanced in syllable units or phoneme units obtained by further subdividing words. Each word is represented by a concatenation of HMM (hidden Markov model) state series for each phoneme.

  FIG. 6 shows an HMM state sequence of the word “Imai”. The phoneme expression of the word “Imai” is “i / m / a / i”, but in general, an HMM state sequence depending on the preceding and following phonemes as shown in FIG. 6 is used to improve the search processing performance. Here, “Sil−i + m” represents an HMM state sequence when the preceding phoneme of the phoneme “i” is “Si1” and the subsequent phoneme is “m”. Each HMM state is allowed to transition to itself (self-transition) and to the right HMM state (LR transition), and the self-transition probability and LR transition probability are described in the acoustic model. The acoustic model describes an output probability distribution for calculating the likelihood (acoustic likelihood) of each acoustic feature quantity obtained for each frame with respect to each HMM state.

  In FIG. 3, the search processing unit 102 calculates the sum (accumulated likelihood) of the transition probability and acoustic likelihood for each of the self transition and the LR transition for every HMM state to be considered in the frame for each frame. , Repeatedly selecting the most likely (highest cumulative likelihood) transition as the HMM state transition, and finally determining the HMM state sequence having the highest cumulative likelihood. Such an algorithm for searching for the maximum likelihood path is called a Viterbi algorithm.

  In such a speech recognition apparatus, in a general-purpose speech recognition application in which a user is not fixed, an unspecified speaker model stored in the acoustic model database 103 is used as an acoustic model. Parameters such as transition probability and output probability distribution described in the unspecified speaker model are learned based on utterance data collected by a large number of speakers. By using an unspecified speaker model, a certain degree of recognition performance can be obtained for any user's utterance.

  On the other hand, by using a speaker adaptation model in which the parameters of the unspecified speaker model are updated with the utterance data collected from a specific speaker, recognition of the utterance of the specific speaker is achieved rather than using the unspecified speaker model. Performance can be improved. Such parameter update is called speaker adaptation, and generally uses a technique such as MAP (Maximum A Postoriori) or MLLR (Maximum Likelihood Linear Regression). For example, when considering a speech recognition application in a mobile phone or the like where the user is likely to be fixed, the use of a speaker adaptation model is effective.

  For speaker adaptation, a set of text representing speech content called speech data and a label is required. This is to update the parameters of the HMM state included in the label based on the utterance data.

  For example, by allowing the user to read (enroll) a word or sentence designated in advance by the voice recognition application, a set in which the utterance data content and the label completely match can be obtained. Speaker adaptation performed using such a set is called supervised adaptation.

  However, enrollment that requires reading out a plurality of words and sentences is troublesome for the user and may hinder speaker adaptation or use of the speech recognition application itself. In addition, general enrollment that is performed only once before using the voice recognition application for the first time cannot cope with changes in user voice over time. That is, even when the user enrolls and the speaker adaptation model is created, the user's voice changes with time, so the deviation from the parameters described in the acoustic model gradually increases. , Recognition performance decreases.

  In response to these problems, the recognition result text by the speech recognition device is used as a label, so that the utterance data and label pair necessary for speaker adaptation can be obtained without forcing the user to enroll. Has been proposed (see, for example, Patent Document 1). Such speaker adaptation is called unsupervised adaptation, and it is possible to perform speaker adaptation each time a speech recognition application is used, so it is possible to cope with changes in user speech over time.

  However, unsupervised speaker adaptation has a problem that the performance of the acoustic model is degraded when there is an error in the recognition result. Therefore, a method using the corrected text as a label has been proposed. FIG. 7 is an example of a conventional speech recognition apparatus that performs unsupervised speaker adaptation using corrected text as a label.

  As in the above-described general speech recognition apparatus, the acoustic analysis unit 101 extracts the acoustic feature amount of the input speech, and the search processing unit 102 obtains the recognition result text. The recognition result text from the search processing unit 102 is sent to the recognition result correction unit 105. The acoustic feature amount from the acoustic analysis unit 101 is stored in the feature amount database 106 as utterance data.

The user confirms the voice recognition result, and if the recognition result is wrong, the user makes corrections via the input device 109 such as a keyboard or a numeric keypad, and the recognition result correction unit 105 corrects the recognition result text. It is done. Then, only the corrected text selected by the user for use in speaker adaptation by the adaptive use selection unit 107 is combined with the acoustic feature amount stored in the feature amount database 106 corresponding to the corrected text, and the speaker adaptation unit 108. Used for speaker adaptation in
JP 2003-241787 A

  However, in the unsupervised adaptation as described above, speaker adaptation is performed by using, as a label, the corrected text obtained by correcting the recognition result text by the user via the input device 109 such as a keyboard or a numeric keypad. However, the corrected text does not necessarily match the utterance data.

  For example, consider a case where text input is performed on a mobile phone using a voice recognition application. Since the user is trying to achieve a certain purpose (search, e-mail, etc.) using the text to be entered, if the recognition result is incorrect, in most cases, the recognition result text is corrected by entering the numeric keypad. it is conceivable that. The user should have uttered the content that he / she wants to input, and it is highly likely that the corrected text and the content of the utterance data match.

  However, the two do not necessarily match. For example, when the user inputs “Mr. Ito” but the voice recognition result is “Mr. Itoi”, the user corrects “Mr. Ito” using the numeric keypad. At this time, there is a possibility of further correction to “Taro Ito”. In this case, while the utterance data is “Mr. Ito”, the corrected text is “Taro Ito”, and the corrected text and the utterance data do not match.

  As described above, since the corrected text and the utterance data do not always match, using the corrected text as a label for speaker adaptation may cause a decrease in performance of the adaptive speaker model.

  In contrast, in the conventional method, whenever the recognition result is corrected, the user is prompted to select whether to use the corrected text for speaker adaptation, and the corrected text is used as a label only when it is selected. Speaker adaptation is performed.

  However, the advantage of unsupervised adaptation is that it does not make the user feel annoyed by the automatic execution of speaker adaptation that accompanies the use of a speech recognition application, which forces the user to select the speaker adaptation execution. That goes against this advantage. In addition, if the memory about the user's utterance data content is ambiguous, it may be used for speaker adaptation, even though the modified text and utterance data content do not match, leading to performance degradation of the adaptive speaker model. There is sex.

  Therefore, the present invention has been made in view of the above-mentioned problems, and by completely determining a set of corrected text and utterance data to be used for speaker adaptation without forcing the user to make a selection, the present invention is completely achieved. An object of the present invention is to provide a speaker adaptation device, a speaker adaptation method, and a speaker adaptation program that can automatically perform speaker adaptation.

The present invention proposes the following items in order to solve the above problems.
(1) The present invention provides an acoustic analysis unit that extracts an acoustic feature of input speech;
A statistical model database in which a statistical model for executing speech recognition based on the extracted acoustic feature quantity is stored; a search processing unit that applies the statistical model to the acoustic feature quantity and executes a search process; and A recognition result correction unit that corrects the recognition result text output by the process from the user, calculates a reliability score for the correction text, and determines whether to use the corrected text for speaker adaptation And a speaker adaptation unit that performs speaker adaptation when it is determined that the corrected text is used for speaker adaptation. .

  (2) According to the present invention, in the speaker adaptation device of (1), the adaptive usage determination unit creates a statistical model for determination with the corrected text as a recognition target, and generates an acoustic feature obtained from the input speech. A speaker adaptation device is proposed that calculates a reliability score based on a cumulative likelihood for a corrected text obtained by executing a search process by applying a statistical model for determination.

  (3) In the speaker adaptation device according to (2), the present invention relates to the frame normalization likelihood obtained by dividing the cumulative likelihood for the modified text by the number of processing frames in the adaptive use determination unit. A speaker adaptation device characterized by a score is proposed.

  (4) In the speaker adaptation device of (2), the present invention is based on the cumulative likelihood for the modified text and the cumulative likelihood for the recognition result text obtained by the search processing unit in the adaptive usage determination unit. A speaker adaptation device that calculates a reliability score is proposed.

  (5) In the speaker adaptation device according to (4), the adaptive use determination unit may include a frame normalization likelihood for the modified text and a cumulative likelihood for the recognition result text obtained by the search processing unit. A speaker adaptation device is proposed that calculates a reliability score based on a frame normalization likelihood with respect to a recognition result text obtained by dividing and by the number of processing frames.

  (6) In the speaker adaptation device according to (5), the adaptive use determination unit calculates a reliability score from a difference in frame normalization likelihood between the corrected text and the recognition result text. A speaker adaptation device is proposed.

  (7) The present invention outputs a first step of extracting an acoustic feature quantity of input speech, a second step of executing a search process by applying a statistical model to the acoustic feature quantity, and the search process. A third step of determining whether or not the user has made corrections to the recognition result text, and calculating a reliability score for the corrected text and whether or not to use the corrected text for speaker adaptation. Proposing a speaker adaptation method comprising: a fourth step for determining a threshold value; and a fifth step for executing speaker adaptation when it is determined that the modified text is used for speaker adaptation. is doing.

  (8) The present invention outputs a first step of extracting an acoustic feature quantity of input speech, a second step of executing a search process by applying a statistical model to the acoustic feature quantity, and the search process. A third step of determining whether or not the user has made corrections to the recognition result text, and calculating a reliability score for the corrected text and whether or not to use the corrected text for speaker adaptation. A speaker adaptation program for causing a computer to execute a fourth step of determining a threshold and a fifth step of executing speaker adaptation when it is determined that the modified text is used for speaker adaptation is proposed. ing.

  According to the present invention, the confidence score for the corrected text is calculated, whether or not the corrected text is used for speaker adaptation is determined as a threshold, and when it is determined that the corrected text is used for speaker adaptation, Perform adaptation. This makes it possible to determine the set of modified text and utterance data to be used for speaker adaptation without forcing the user to select the speaker, and to perform speaker adaptation completely automatically without making the user feel bothered There is an effect that can be. In addition, in order to determine whether or not to use speaker adaptation without relying on the user's account for the utterance content, it is necessary to more accurately select a combination of the corrected text and utterance data content and use it for speaker adaptation. With this, the performance of the adaptive speaker model can be improved.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Note that the constituent elements in the present embodiment can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

FIG. 1 shows functional blocks of the present embodiment.
In FIG. 1, an input voice signal is sent to an acoustic analysis unit 11, and an acoustic feature quantity is extracted from the input voice by the acoustic analysis unit 11.

  The acoustic feature amount extracted by the acoustic analysis unit 11 is sent to the search processing unit 12. The acoustic feature amount is stored in the feature amount database 16 as utterance data for speaker adaptation processing.

  The acoustic model database 13 and the language model database 14 store statistical acoustic models and language models created in advance. The search processing unit 12 performs search processing according to statistical acoustic models and language models in the acoustic model database 13 and the language model database 14 based on the acoustic feature amount, and a speech recognition result is output from the search result. The Note that these acoustic analysis processing and search processing are the same as those of a general voice recognition device.

  The user confirms the voice recognition result, and if the recognition result is incorrect, the user makes correction via the input device 19 such as a keyboard or a numeric keypad. The recognition result correction unit 15 corrects the recognition result text from the user. Then, the corrected text is sent to the adaptive usage determination unit 17.

  The adaptive usage determination unit 17 executes a search process for the acoustic feature quantity of the input speech obtained by the acoustic analysis unit 11 using the determination language model created from the modified text obtained by the recognition result modification unit 15 and accumulates the modified text. Calculate the likelihood. Based on this, the reliability score of the corrected text is calculated, and if the reliability score satisfies the condition based on a predetermined threshold, it is determined that the corrected text matches the content of the input speech and can be used for speaker adaptation. To do.

  In other words, the adaptive use determination unit 17 first creates a language model (determination language model) whose recognition target is only the content from the obtained corrected text. Since this is created depending only on the corrected text, it may be a word or a word string that is not a recognition target in the language model used in the search processing unit 12. Next, using the acoustic model and language model for determination in the acoustic model database 13 and the language model database 14, a corresponding acoustic feature amount search process is executed, and the cumulative likelihood for the corrected text is calculated.

  Here, since there are only one word or word string to be recognized in the determination language model, the recognition result always matches the corrected text. When the reliability score is calculated from the calculated cumulative likelihood and the reliability score satisfies a predetermined threshold value, it is determined that the corrected text matches the utterance data content and can be used for speaker adaptation. The

  In addition, the processing in the adaptive usage determination unit 17 can set the entire corrected text or only a part thereof as a processing target.

  As the reliability score, for example, it is conceivable to use a frame normalization likelihood obtained by dividing the calculated cumulative likelihood by the number of frames processed in the search process. This is because the cumulative likelihood itself increases as the number of processing frames increases, and is not suitable for threshold determination.

  It is also conceivable to calculate the reliability score by using not only the cumulative likelihood of the corrected text but also the cumulative likelihood calculated when the search processing unit 12 obtains the recognition result. For example, the reliability score based on the frame normalization likelihood for the corrected text and the frame normalization likelihood for the recognition result text obtained by dividing the cumulative likelihood for the recognition result text obtained by the search processing unit 12 by the number of processing frames. Is calculated.

  Next, when the adaptive usage determination unit 17 determines that the reliability score satisfies the condition based on a predetermined threshold, the speaker adaptation unit 18 executes speaker adaptation processing. In the speaker adaptation process, a process for adapting the acoustic model in the acoustic model database 13 to the speaker by combining the corrected text as a label and the corresponding acoustic feature quantity stored in the feature quantity database 16 as utterance data. Done.

  As described above, in this embodiment, the acoustic feature amount of the input speech obtained by the acoustic analysis unit 11 by the adaptive use determination unit 17 by the determination language model created from the corrected text obtained by the recognition result correction unit 15. , The cumulative likelihood of the corrected text is calculated, and the confidence score of the corrected text is calculated based on this. When the reliability score satisfies the condition based on a predetermined threshold, speaker adaptation is performed. The unit 18 executes speaker adaptation processing. For this reason, it is possible to correctly determine a set of corrected text and utterance data used for speaker adaptation.

  For example, in the case where the voice recognition result is “Mr. Itoi” even though the user has input the voice “Mr. Ito”, the user uses the input device 19 to change the recognition result correction unit 15 to “ Ito-san ". In addition, at this time, there is a possibility of further correcting as “Taro Ito”. When the correction is made with “Mr. Ito”, the cumulative likelihood of the corrected text becomes larger than the threshold value, so the speaker adaptation process is executed. On the other hand, when the correction is made as “Taro Ito”, the cumulative likelihood of the corrected text is smaller than the threshold value, so the speaker adaptation process is not executed.

  Thus, in this embodiment, without forcing the user to select, by determining the combination of corrected text and utterance data to be used for speaker adaptation, the user is completely annoyed without feeling bothered. Speaker adaptation can be performed automatically. In addition, in order to determine whether or not to use speaker adaptation without relying on the user's account for the utterance content, it is necessary to more accurately select a combination of the corrected text and utterance data content and use it for speaker adaptation. Thus, the performance of the adaptive speaker model can be improved.

  The present invention can be realized by hardware that performs processing equivalent to that shown in FIG. 1 or software equivalent thereto. FIG. 2 shows a flowchart of the embodiment of the present invention.

  In FIG. 2, when a voice is input (step S1), an acoustic feature amount of the input voice is extracted (step S2). Further, the extracted acoustic feature amount is stored as utterance data (step S3). Then, based on the acoustic feature quantity, search processing is performed according to a statistical acoustic model and a language model (step S4), and a speech recognition result is output from the search result (step S5).

  It is determined whether or not the correction result text has been added to the recognition result text (step S6). If no correction has been made, the process returns to step S1. If correction has been made, a determination language model is created from the corrected text (step S7). And the search process of the acoustic feature-value of input speech is performed with the created language model for determination, and the reliability score of the corrected text is calculated based on the cumulative likelihood of the corrected text (step S8). It is determined whether or not the reliability score is larger than a predetermined threshold value (step S9). If the reliability score is smaller than the threshold, the process returns to step S1. If the reliability score is larger than the threshold value, speaker adaptation processing is executed (step S10), and the process returns to step S1.

  The source program is provided on a computer-readable recording medium such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM. The source program may be transmitted from a computer system to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The source program may be a program for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the gist of the present invention.

It is a block diagram of one embodiment of the present invention. It is a flowchart used for description of one Embodiment of this invention. It is a block diagram of the basic composition of a speech recognition device. It is explanatory drawing of the frame process of acoustic analysis. It is explanatory drawing of a fixed grammar model. It is explanatory drawing of a front and back phoneme environment dependence HMM state. It is a block diagram of an example of the speech recognition apparatus which performs the conventional unsupervised speaker adaptation.

Explanation of symbols

11: acoustic analysis unit 12: search processing unit 13: acoustic model database 14: language model database 15: recognition result correction unit 16: feature amount database 17: adaptive use determination unit 18: speaker adaptation unit 19: input device

Claims (8)

An acoustic analysis unit for extracting acoustic features of the input speech;
A statistical model database in which a statistical model for executing speech recognition based on the extracted acoustic features is stored;
A search processing unit that executes the search process by applying the statistical model to the acoustic feature amount;
A recognition result correction unit for correcting the recognition result text output by the search process from the user;
An adaptive usage determining unit that calculates a reliability score for the corrected text and determines whether or not to use the corrected text for speaker adaptation;
A speaker adaptation unit that performs speaker adaptation when it is determined that the modified text is used for speaker adaptation;
A speaker adaptation device characterized by comprising:   A correction obtained by executing a search process by creating a statistical model for determination using the corrected text as a recognition target and applying the statistical model for determination to the acoustic feature obtained from the input speech in the adaptive usage determination unit The speaker adaptation device according to claim 1, wherein a confidence score is calculated based on a cumulative likelihood for the text.   3. The speaker adaptation according to claim 2, wherein the adaptive use determination unit uses a frame normalization likelihood obtained by dividing the cumulative likelihood for the modified text by the number of processing frames as a reliability score. apparatus.   The said adaptive use determination part calculates a reliability score based on the cumulative likelihood with respect to the said correction text, and the cumulative likelihood with respect to the recognition result text obtained in a search process part, The feature score of Claim 2 characterized by the above-mentioned. Speaker adaptation device.   In the adaptive use determination unit, the frame normalization likelihood for the recognition result text obtained by dividing the frame normalization likelihood for the modified text and the cumulative likelihood for the recognition result text obtained by the search processing unit by the number of processing frames. 5. The speaker adaptation apparatus according to claim 4, wherein a reliability score is calculated based on the degree.   6. The speaker adaptation apparatus according to claim 5, wherein the adaptive use determination unit calculates a reliability score from a difference in frame normalization likelihood between the modified text and the recognition result text. A first step of extracting an acoustic feature of the input speech;
A second step of executing a search process by applying a statistical model to the acoustic feature amount;
A third step of determining whether or not correction from the user has been made to the recognition result text output by the search process;
A fourth step of calculating a reliability score for the corrected text and determining whether or not to use the corrected text for speaker adaptation;
A fifth step of performing speaker adaptation if it is determined to use the modified text for speaker adaptation;
A speaker adaptation method characterized by comprising: A first step of extracting an acoustic feature of the input speech;
A second step of executing a search process by applying a statistical model to the acoustic feature amount;
A third step of determining whether or not correction from the user has been made to the recognition result text output by the search process;
A fourth step of calculating a reliability score for the corrected text and determining whether or not to use the corrected text for speaker adaptation;
A fifth step of performing speaker adaptation if it is determined to use the modified text for speaker adaptation;
A speaker adaptation program that causes a computer to execute.

Images