JPH09114482A - Speaker adaptation method for voice recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an acoustic model speaker-adaptation without a teacher. SOLUTION: Recorded recognition object voices of a speaker is tried to be recognized employing an un-specified speaker model (S1 and S2 ). Then, the recognition results are used as teacher signals in making the above unspecified speaker model adaptive to recognition object voices (S3 and S4 ). After that, the adaptive model is used to re-recognize the recognition object voices (S5 ). If the recognition results are the same as in a previous case or do not exceed a limiting time, the process goes back to the step S3 . If the results are the same or exceed the limiting time, the process stops (S6 ).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speaker adaptation method for speech recognition, which is suitable when it can be done offline for a sufficient period of time. For example, like a dictation of a medical finding, it can be used for efficiency of work such as writing down and documenting a diagnostic result recorded by a doctor uttered on a tape by a professional worker.

[0002]

2. Description of the Related Art FIG. 3A shows a speech recognition process using a phoneme as a unit of recognition. In FIG. 3A, the input speech 11 is converted into a time series of characteristic parameters by the analysis processing 12, and the time series of the characteristic parameters is converted by the search processing 13 into the grammar 1
The matching with the phoneme model 15 is performed while using the constraint condition of 6. Then, the phoneme sequence having the highest evaluation value is output as the recognition result 14.

Linear prediction analysis (Linear Predictive Analysis) is often used as the signal processing in the analysis processing 12.
Coding, LPC), and the characteristic parameters include LPC cepstrum, LPC delta cepstrum, mel cepstrum, and logarithmic power. As the phoneme model 15, a Hidden Markov Model method (hereinafter referred to as HMM method) which is modeled based on probability / statistical theory is mainstream. Details of the HMM method are disclosed in, for example, "Speech Recognition by Stochastic Model" by Seiichi Nakagawa, edited by The Institute of Electronics, Information and Communication Engineers.

The search processing 13 evaluates the plausibility of the hypothesis corresponding to the phoneme string permitted by the grammar with the input speech, and advances the search while expanding the phonemes into the hypotheses one by one. Here, the hypothesis means a phoneme string connected according to the restriction of the order of arrangement of phonemes shown in the grammar, and the extension of the phoneme to the hypothesis means one more phoneme in the phoneme string of the hypothesis according to the grammar. Means to connect.

A case where the phoneme model 15 used for speech recognition is made from the speech of the speaker to be recognized is called specific speaker recognition, and is usually made from the speech of one or more speakers other than the speaker to be recognized. Is called unspecified speaker recognition. Comparing the recognition performance, the specific speaker recognition in which the phoneme model 15 is created from the voice of the recognition target speaker is generally higher. In order to create the phoneme model 15, it is usually necessary to prepare a voice that includes all vocabulary to be recognized. Therefore, when the number of vocabulary to be recognized is large, it is difficult to utter such an advance voice.
The unspecified speaker recognition is performed, or the speaker adaptation described below is performed on the phoneme model 15 of the unspecified speaker using a small amount of voice of the recognition target speaker, and then the recognition is performed.

Speaker adaptation is a technique in which the phoneme model 15 used for unspecified speaker recognition is obtained by using the voice of the recognition target speaker to obtain a phoneme model 15 close to the phoneme model 15 used for specific speaker recognition. is there. In general, for the speaker adaptation, the speech of the recognition target speaker whose utterance content is known is used. In this way, the process of teaching the utterance contents to perform the speaker adaptation is called the speaker adaptation with a teacher, and the speaker adaptation that gives the voice of which the utterance contents are unknown is called the unsupervised speaker adaptation.

Conventional speaker adaptation is classified into the following three types.・ Batch adaptation Before using for recognition of unknown speech, a special voice was recorded in advance for speaker adaptation separately from the recognition target, and speaker adaptation was performed as a batch process using this and adapted to that speaker. Create an acoustic model. It is general that the utterance content of the voice recorded in advance is known. The obtained acoustic model is used to recognize the unknown voice of the speaker.

Sequential adaptation Unlike batch adaptation, when performing voice recognition, recording of voice used for speaker adaptation and speaker adaptation processing are not performed in advance. The recognition target voices that are sequentially input online are recognized each time they are input, and the process of performing speaker adaptation is repeated. Each time recognition is performed for one unit (word, phrase, etc.), speaker adaptation is performed on the acoustic model based on the recognition result or the teacher signal obtained by checking / correcting it, and new adaptation is performed. A model is created and used for subsequent speech recognition.

Immediate adaptation The voice for speaker adaptation is not recorded in advance, and the speaker adaptation processing is not performed. In batch adaptation and sequential adaptation, the speech used for adaptation is different from the speech that is the recognition target of the model obtained by adaptation, but in immediate adaptation, the speech itself is used for adaptation. Speaker adaptation is performed for each voice input, and the input model is recognized by the obtained model. Since the adaptation method focuses on the characteristics of the spectrum of the voice, whether the utterance content is unknown or known is irrelevant to the adaptation.

[0010]

The above-mentioned conventional methods have the following problems, respectively. -Batch adaptation. The speech used only for speaker adaptation not related to recognition shall be recorded. Since the feature of the voice including all the voices to be recognized cannot be reflected in the model by the speaker adaptation, the adaptation effect may be lowered.

Sequential adaptation Since the speaker adaptation is performed from the partial voice, the recognition accuracy cannot always be improved by the adaptation method. Since the speech used for speaker adaptation does not necessarily include the speech to be recognized, its speaker nature cannot be sufficiently reflected in the model, and the adaptation effect may not be fully utilized.

Immediate adaptation Since the speaker adaptation is performed from a partial voice, the recognition accuracy is not always improved by the adaptation method. It is reported that the improvement of the recognition rate is small when there is no significant difference between the speech of the unspecified speaker model and the speech of the recognition target speaker.

[0013]

According to the present invention, accumulated recognition target speech is recognized using an acoustic model, and the recognition result is used as a teacher signal to adapt the acoustic model to the recognition target speech. We effectively utilize the given time without human intervention, and obtain a high recognition rate for the speech to be recognized. The procedure of the invention of claim 2 is as follows.

The first step of recognizing the speech of the speaker to be recognized using the initial acoustic model, and the second step of performing speaker adaptation of the acoustic model using the recognition result obtained by this recognition as a teacher signal, The third step of recognizing the voice using the new acoustic model obtained by adaptation, and if the recognition result is the same as the previous time or if the time limit is exceeded, end, otherwise go to the second step move on. With this configuration, it is repeated off-line for a sufficient time, and the personality information peculiar to the recognition target speaker existing in the recognition target speech itself is directly reflected in the acoustic model. It is possible to obtain a speaker-adapted acoustic model based on individuality information obtained from the entire recognition target speech, rather than speaker adaptation from partial speech.

According to the third aspect of the present invention, the recognition target speech is recognized as an acoustic model, a part of the recognition result is confirmed / corrected, and the recognition target speech of the portion for which confirmation / correction is completed is
The confirmation / correction result is used as a teacher signal, and the recognition target speech of the part for which confirmation / correction is not completed is used as a teacher signal using the previous recognition result as it is, and speaker adaptation is performed on the acoustic model to be recognized. A new acoustic model adapted to the speaker is created, and the recognition target speech is recognized again using this new acoustic model.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the configuration of the invention of claim 2. 1. SW2 is switched to A and the unspecified speaker model 21 is loaded for the recognition processing unit 25 (S ₁ ). 2. The recognition target speech 22 (for example, recorded on a tape in a medical finding) is subjected to the recognition processing 25 for each clause (S ₂ ). 3. SW1 is switched to C and the unspecified speaker model 21 is used as an initial model for speaker adaptation. 4. Speaker adaptation processing 23 is performed using the above recognition result as a teacher signal (S ₃ ). For this adaptive processing, a maximum likelihood estimation method, a maximum posterior probability estimation method, or a method combining this with a motion vector field smoothing method is used. 5. A new acoustic model 24 obtained by the adaptation switches the SW2 to B load for recognition processing 25 (S _4). 6. The recognition target speech 25 used previously is subjected to the recognition processing 25 for each phrase (S ₅ ). 7. Recognition result determines exceeded same or a time limit as before (S _6). 8. If the recognition result is not the same as the previous time and the time limit has not elapsed in this determination, SW1 is switched to D, and the acoustic model 24 obtained by the previous adaptation is set as the initial model for speaker adaptation, and the process proceeds to 4. That is, the process returns to step S ₃ . When the recognition result is the same as the previous one or when the time limit is exceeded, the process ends.

As described above, since the speaker is applied to the acoustic model using the recognition result as a teacher signal and the same recognition target speech is recognized again by the applied acoustic model, the repetition proceeds. As a result, the speaker adaptation is improved and the recognition rate is improved. Next, an experimental example will be described. The speaker adaptation according to the present invention and the conventional speaker adaptation method are compared and evaluated by using the recorded voice concerning the findings of the head X-ray CT. In this paper, we introduced a measure based on the total correction man-hours of recognition errors in documenting the entire speech to be recognized as a measure for evaluating the effectiveness of speech recognition in dictation. One is the ratio of the bunsetsu that does not need to correct the recognition result in the entire recognition target, and this is called the bunsetsu recognition rate. In addition, we also compared the total correction man-hours, which weighed the corrections of erroneous recognition with the number of operations.

For speaker adaptation, an HMM is formed by concatenating syllable utterances and their phoneme label sequences without cutting out syllable utterances.
Connected learning is used. Even when parameter estimation is performed using all speech to be recognized, it is difficult to secure parameter estimation accuracy due to the small number of samples depending on phonemes, so adaptation was performed using only the average vector.
As the adaptive algorithm, a maximum likelihood (ML: Maximum Likilihood) estimation method using a Baum-Welch algorithm and a maximum posterior probability (MAP: Maximum a Posteriori) estimation method are used.

An acoustic model of an unspecified speaker is created by using ATR important word 5240, phoneme balance word 216 for each of 10 males and 10 females, and 150 phoneme balance sentences of ASJ database for 30 males and 34 females. did. The speech used in the recognition experiment was a head X-ray CT finding that two female speakers uttered at each phrase at a natural speed, and the volume was 30 finding each (1
300 bunsetsu, 14000 phonemes). This is generally the standard daily dose for a radiologist.

The acoustic analysis condition of voice is a sampling frequency of 12K.
Hz, Hamming time window, analysis window length 32 ms, 16th-order LPC analysis, features are 16th-order LPC cepstrum, 16th-order Δ cepstrum, and Δ-power. The HMM used in the experiment is an environment-dependent HMM, the number of basic phonemes is 26, and the number of environment-dependent models is about 1700. The structure is HM net with 450 states, and the output probability is a 4-mixed continuous distribution.

As a comparison target, batch speaker adaptation in which a teacher signal is given to all recognition target speeches for which it is considered that a result close to that in the case where optimum adaptation is performed is obtained.
Alternatively, a teacher signal was given to the recognition result for each sentence, and based on this, the speaker adaptation was performed using the MAP estimation method to recognize the speaker adaptation and the unspecified speaker model. Through inputting all the findings, the phrase recognition rate is calculated with the number of erroneous recognition phrases that the user has to correct as an evaluation criterion. In this experiment, the phrase recognition rate was calculated by comparing phoneme sequences.

Sequential speaker adaptation For each sentence or finding, 1. Recognition is performed using the adaptation model obtained up to that point. However, the first sentence (finding) uses an unspecified model. 2. The total number of phrases in one sentence (finding) and the number of correctly recognized phrases are counted. 3. The teacher model is given and the acoustic model is adapted.

By repeating the above process, the phrase recognition rate is calculated from the total number of phrases and the total number of correct sentences after the process of all findings is completed. In the case of repeated unsupervised speaker adaptation (the present invention), the bunsetsu recognition rate is calculated from the total number of bunsetsus that have been recognized after unsupervised adaptation of all findings and the total number of correctly bunsetsus.
FIG. 3B shows the relationship between the average phrase recognition rate of two speakers and the number of repetitions. When repeated unsupervised speaker adaptation is performed using the MAP estimation method (MAP of the present invention), recognition rate is 86.3% at the third iteration, and repeated unsupervised speaker adaptation is performed using the ML estimation method. (Invention ML) is repeated 4
The recognition rate was 85.3% at the first time. The recognition rate is 94.1% when batch speaker adaptation (batch adaptation) is performed by giving a correct phoneme sequence as a teacher signal, and when sequential speaker adaptation (sequential adaptation) is performed by the MAP estimation method for each sentence. 90.5%
The average recognition rate was 98.7% for each finding, and the average recognition rate was 78.4% for phrase recognition using the unspecified speaker model (unspecified speaker). In the method of the present invention using the MAP estimation method, the erroneous recognition rate of the bunsetsu was reduced from 21.6% to 13.7% after three repetitions, and the erroneous recognition improvement rate was 37%.

In a general dictation system, only the 1st place of the recognition result is displayed on the display,
The second place and below are displayed only when some operation is performed. If a large number of second and lower candidates are displayed, the probability that a correct recognition result is included increases, but if the number of displayed candidates is too large, similar candidates are lined up, and it becomes difficult to select a correct candidate. For this reason, it is general to display candidates up to the 10th place. From such a fact, in the dictation system, the recognition result can be classified into the following three types.

The one in which the first place is the correct answer.・ Those who have the correct answers from 2nd to 10th.・ There is no correct answer within the 10th place. Considering the normal usage pattern, the operation required to correct the recognition result of the second group is to instruct the system to display the next candidate and select the correct candidate from the second to tenth recognition results. There are two operations to do. Therefore,
Correcting this group of errors requires two operations each. Similarly, the man-hour required to correct the recognition result of the third group is to instruct the system to display the next candidate, to inform the system that there is no correct conversion result in the next candidate, and to convert kana to kanji ( In this case, it is assumed that roman characters are input), n characters are input, kana-kanji conversion is instructed, and character string confirmation is instructed. Therefore, the error correction of this group is
For each, 4 + n operations are required. Usually, a part of the character string of the recognition result can be used, and it is possible to press the "next candidate key" several times when performing Kana-Kanji conversion, but the calculation of the number of operations is simplified according to the above method. Went to. In order to clarify the relationship with the recognition rate, the number of operations per 100 phrases required to correct the recognition result is defined as a correction index. By doing this, the man-hours required to correct the recognition result are compared. The correction index is shown in FIG. 3C. MAP according to this invention
When the unsupervised speaker adaptation was repeated three times using the estimation method, the correction index decreased by 36% as compared with the case of recognition using the unspecified speaker model.

According to the method of the present invention, a large performance improvement was obtained by the first adaptation in both MAP estimation and ML estimation. This is considered to be because the effect of speaker adaptation approaching the model of the target speaker from the acoustic model of the unspecified speaker is greater than the effect of the wrong teacher signal. It is considered that the repetition of two or more times does not improve the performance because the effect of speaker adaptation is smaller than the effect of the wrong teacher signal and the same misrecognition is repeated.

Looking at the relationship between the erroneous recognition rate and the correction index in the evaluated adaptation system, the modification index having a lower recognition rate increases the correction index at a rate higher than that of the erroneous recognition. The false recognition rate indicates that the first recognition result is not the correct answer, whereas the modified index is an evaluation criterion that takes into account not only the number of false recognitions but also the rank of correct answer candidates. . From the above, it can be considered that the correction index is more suitable as a scale for evaluating the performance of the dictation system from the viewpoint of the user than the misrecognition rate.

Although the present invention has a correction index about twice as large as that of the instant recognition using the sequential adaptation, the voicing and the documentation are division of labor, so from the viewpoint of workability,
This doesn't seem to matter. The speaker does not have to wait for the recognition result and confirm / correct the recognition result each time he / she speaks, so there is no interruption of thinking, and the transition from the conventional method of recording the utterance on tape can be performed easily. This is because the person who performs the conversion can produce a document that is correctly converted by about 87% without human intervention.

Next, an embodiment of the invention of claim 3 will be described with reference to FIG. In this embodiment, the speech to be recognized is recognized by the unspecified speaker acoustic model (S ₁ ), the recognition result is confirmed / corrected (S ₂ ), and the confirmation / correction is temporarily performed for eating out, for example. interrupted (S _3). During the interruption, in the present invention, for the recognition target voice of the portion whose confirmation / correction is completed, the result of the confirmation / correction is used as a teacher signal, and for the recognition target voice of the portion whose confirmation / correction is not completed, step S The recognition result in ₁ is used as a teacher signal as it is for speaker adaptation to the acoustic model and adapted to the speaker to be recognized to create a new acoustic model (S ₄ ).
Next, the speech to be recognized is recognized again using this new acoustic model (S ₅ ), and it is checked whether the recognition result is the same as the previous time or whether the time limit has been exceeded. If not, the process returns to step S ₄ to recognize the recognition result. If the same or the time limit is exceeded, the process ends (S ₆ ). In this case, the recognition in step S ₅ may be performed only on the recognition target voice of the part for which confirmation / correction is not completed.

In the learning shown in FIG. 2, the same experiment as described above was conducted, and the first half of the data used in the experiment was 1/3.
An experiment was conducted by the above-mentioned adaptive method using 0 observation as a teacher's voice, that is, a confirmed / corrected part of the voice, and an unspecified speaker model as an initial model. As a result of evaluation based on "the number of erroneous recognition phrases that the user must correct through inputting all findings", the recognition rate was 84.4% in the first iteration by the MAP estimation method and 83.3% by the ML estimation method. It was 9%. In the aggregation of all findings, the recognition rate seems low because it includes the number of man-hours for correcting the findings in the first half of the recognition recognized by the unspecified speaker model. When 20 tabulations in the latter half 2/3 are tabulated to confirm the effect of adaptation, the MAP in the invention of claim 2 is 86.6.
% And ML were 85.3%, 87.2% and 86.
It improved to 3%. From above, teacher signal (confirmation / correction signal)
Shows that the model performance is improved.

In the above two embodiments, the unspecified speaker acoustic model is used as the initial model, but an acoustic model created from the voice uttered by the recognition target speaker may be used. Also in this case, it is difficult to increase the amount of speech for creating the acoustic model so much, and therefore, the speech to be recognized may include a large amount of speech other than the speech used for creating the acoustic model. Therefore, the present invention is effective. Further, as is clear from the above experimental example, once the speaker adaptation of the acoustic model using the recognition result as the teacher signal is performed, the recognition rate is remarkably improved, and the speaker adaptation and the recognition are not repeated. Good.

[0032]

As is apparent from the above description, according to the present invention, the voice recognition rate for the recognition target voice is improved by the adaptive processing without human intervention as compared with the unspecified speaker model, and it is erroneously recognized. This has the effect of significantly reducing the effort required to correct the results. Each 30 uttered by two speakers
As a result of applying this method to sentences (2632 phrases), the false recognition rate was reduced by about half.

In a normal dictation device, the next candidate is generally the second to tenth recognition result displayed. Therefore, if the correct recognition result is in the 2nd to 10th positions, it is corrected by the operation of giving an instruction to display the next candidate on the screen of the device and the operation of selecting one from the displayed candidates. The work is completed. If there is no correct recognition result within the 10th place, the operation will be the same as a normal Japanese word processor, so to correct the incorrect recognition result, the operation to display the next candidate was not from 2nd to 10th place. To inform the device, as in the case of a word processor, to input the correct text in Roman characters, it is necessary to operate as many times as the number of characters, to perform Kana-Kanji conversion, and to confirm the conversion result. Comparing the labor required for correction by the number of operations required to manually correct the misrecognized conversion results counted in this way, the conventional method requires about 1600 operations. What was present was reduced to about 1100 operations using this method.

[Brief description of the drawings]

1A is a block diagram showing an embodiment of the invention of claim 3; FIG. 1B is a flow chart showing a processing procedure of an embodiment of the invention of claim 2;

FIG. 2 is a flowchart showing a processing procedure of an embodiment of the invention of claim 3;

FIG. 3A is a block diagram showing a speech recognition process using a phoneme as a unit of recognition, and B shows an experimental example of the relationship between the number of repetitions and the phrase recognition rate of the invention of claim 2, and the recognition rate of the conventional method. FIG. 6C is a diagram showing an experimental example of the correction index of the method of the invention of claim 2 and the conventional method.

Claims (5)

[Claims] 1. A first step of recognizing a recorded recognition target voice uttered by the same speaker using an acoustic model, and speaker adaptation using the recognition result obtained in the first step as a teacher signal, And a second step of creating a new acoustic model adapted to the speech to be recognized, the speaker adaptation method for speech recognition. 2. A third step of recognizing the recognition target speech again using the newly created acoustic model, and a recognition result obtained in the third step as a teacher signal for the new acoustic model. Until the recognition result becomes the same as the last time, the fourth step of performing speaker adaptation to create a new acoustic model further adapted to the speaker to be recognized, and the third step and the fourth step, or A speaker adaptation method for speech recognition, comprising a fifth step of repeating until a predetermined processing time is exceeded. 3. A first step of recognizing a speech to be recognized using an acoustic model prepared in advance, a second step of confirming / correcting a part of the recognition result obtained in the first step, and a second step thereof. The result of confirmation / correction is used as a teacher signal for the recognition target voice of the part whose confirmation / correction is completed in the two steps, and the recognition target voice of the part for which confirmation / correction is not completed is obtained in the first step. Using the recognition result as it is as a teacher signal, speaker adaptation is performed on the acoustic model to create a new acoustic model adapted to the speaker to be recognized. A fourth step of re-recognizing the speech to be recognized using the acoustic model obtained, and a speaker adaptation method for speech recognition. 4. The initial acoustic model used in the first step is an acoustic model created from speech uttered by one or more speakers other than the recognition target speaker. Adaptation Method for Speech Recognition in Humans. 5. The speaker for speech recognition according to claim 2 or 3, wherein the acoustic model used in the first step is an acoustic model created from speech uttered by a recognition target speaker. Adaptation method.