JP4951035B2 - Likelihood ratio model creation device by speech unit, Likelihood ratio model creation method by speech unit, speech recognition reliability calculation device, speech recognition reliability calculation method, program - Google Patents

Description

  The present invention relates to a technique for creating a likelihood ratio model for each speech unit and calculating the reliability of speech recognition using the likelihood ratio model.

  In the speech recognition technology, since the reliability of the speech recognition result is calculated and the correctness / incorrectness of the speech recognition result may be determined based on this reliability, such reliability is one of important indexes.

  For example, Non-Patent Document 1 is given as a reliability calculation method using the likelihood of a syllable recognition result (syllable likelihood) as a reference likelihood. In this technique, as shown in FIG. 1, a feature amount analysis unit 901 obtains an acoustic feature amount of a digital audio signal for each frame, and a recognition processing unit 902 uses an acoustic model 903 and a dictionary / language model 904. Speech recognition processing is performed on the acoustic feature amount for each frame to calculate a speech recognition result and its recognition likelihood (task likelihood), and the syllable likelihood calculating unit 905 obtains the syllable likelihood from the acoustic feature amount for each frame, The reliability calculation unit 906 obtains the reliability by normalizing the task likelihood with the syllable likelihood. As described above, this technique normalizes the task likelihood based on speech recognition using the recognition target dictionary with the syllable likelihood based on syllable recognition, and enhances the correctness / incorrectness discrimination ability in the input outside the vocabulary.

  Further, in Patent Document 1, the recognition likelihood of the word with the highest recognition likelihood among the words with the recognition likelihood (score) different from the word with the first rank and the second rank or higher, and the first rank A reliability calculation method based on N-best is disclosed, in which a likelihood difference with the recognition likelihood is used as a reference likelihood.

  Also, Julius (see Non-Patent Document 2) and the like exist as open source general-purpose large vocabulary continuous speech recognition engines for developing speech recognition systems.

Japanese Patent No. 3819896

Takao Watanabe, Satoshi Tsukada, "Rejection of unknown utterances by likelihood correction using syllable recognition", IEICE Transactions D Vol.J75-D2 No.12 pp.2002-2009, 1992. Julius, [Search July 2, 2009], Internet <http://julius.sourceforge.jp/index.php>

  According to the technique disclosed in Non-Patent Document 1, since the calculation process of syllable likelihood is accumulated, the processing amount of the entire recognition process is large.

  According to the technique disclosed in Patent Document 1, although the reference likelihood is obtained by obtaining N-best, the calculation processing is small, but it cannot be said that the correctness / incorrectness discrimination ability is necessarily good for an out-of-vocabulary input.

  In addition, since the calculation of reliability in the prior art depends on the frame length and the number of frames, the range of recognition likelihood and reference likelihood is not determined. For this reason, it is difficult to universally normalize the reliability (for example, fall within a value of 0 to 100) regardless of the linguistic unit (for example, word) and the utterance time.

  In view of such a situation, the present invention creates a normalized likelihood ratio model for each speech unit and calculates each technique for calculating the reliability of speech recognition normalized using this likelihood ratio model. The purpose is to provide.

  The present invention relates to a technique for creating a normalized likelihood ratio model for each speech unit, an acoustic model, development data composed of speech data and correct labels associated with the speech data, and a mixed normal distribution. (GMM) is used to calculate the acoustic feature amount of the voice data for each frame (feature amount analysis), and the correct feature label and the acoustic model are used for the acoustic feature amount for each frame and included in the correct label. The correct likelihood of each speech is calculated (correct likelihood calculation), and the likelihood by GMM (GMM likelihood) is calculated for each frame acoustic feature (GMM likelihood calculation). The ratio between likelihood and likelihood of GMM (likelihood ratio for each voice unit) is calculated (likelihood ratio for each voice unit), and for each voice unit included in the development data, the likelihood ratio for each voice unit corresponding to the voice unit Probability distribution with a random variable The number to create a normalized normalized probability distribution function (audio unit by the likelihood ratio model) (by speech unit likelihood ratio modeling).

  This GMM is assumed to be a mixed normal distribution learned from the voiced interval of the learning speech data, and the GMM likelihood is calculated using the unvoiced model learned from the unvoiced interval of the learning speech data. You may make it calculate.

  Further, the present invention relates to a technique for calculating the reliability of speech recognition normalized using this likelihood ratio model, an acoustic model, the above-described likelihood unit model for speech units, and a mixed normal distribution ( GMM) is used to calculate the acoustic feature amount of the speech signal to be recognized for each frame (feature amount analysis), and for the acoustic feature amount for each frame, using the acoustic model, The recognition result likelihood of the speech unit included in the speech recognition result is calculated (recognition processing), the likelihood (reference likelihood) by GMM is calculated for the acoustic feature amount for each frame (reference likelihood calculation), For each frame, the ratio of the recognition result likelihood to the reference likelihood (likelihood ratio for each voice unit) is calculated (likelihood ratio for each voice unit), and for each voice unit included in the voice recognition result, Likelihood ratio by speech unit for each corresponding frame Output value of the audio unit by the likelihood ratio model when a force determined (frame reliability), respectively, the frame average value of the frame confidence determined as a speech unit reliability (speech unit reliability calculation).

  The average value of the speech unit reliability corresponding to each speech unit included in the speech recognition result may be calculated (reliability calculation) as the reliability of the speech recognition result.

  Regarding the reliability calculation technique, the acoustic model is the same acoustic model as that used when creating the speech unit likelihood ratio model, and the GMM used when creating the speech unit likelihood ratio model Is preferably the same GMM.

  Further, the computer can be operated as a speech unit likelihood ratio model creation device by a program that causes the computer to function as the speech unit likelihood ratio model creation device of the present invention. Similarly, the computer can be operated as a speech recognition reliability calculation device by a program that causes the computer to function as the speech recognition reliability calculation device of the present invention. Although the details will be described in the embodiment, it is also possible to realize the speech unit likelihood ratio model creation device and the speech recognition reliability calculation device as a single device. The computer can be operated as a speech unit-specific likelihood ratio model creation device and a speech recognition reliability calculation device by a program written to cause the computer to function as the degree ratio model creation device and the speech recognition reliability calculation device. .

  According to the present invention, a normalized likelihood ratio model for each speech unit is created, and the reliability of speech recognition normalized using this likelihood ratio model is calculated. Regardless of this, universally normalized reliability can be obtained, and its handling is convenient.

The figure explaining the conventional reliability calculation technique which uses a syllable recognition result likelihood (syllable likelihood) as a reference likelihood. The figure which shows the function structural example which implements the likelihood ratio model preparation process classified by phoneme of embodiment by this invention. The figure which shows the processing flow which implements the likelihood ratio model preparation process classified by phoneme of embodiment by this invention. The figure which shows the specific example of the likelihood ratio model preparation process classified by phoneme of embodiment by this invention. The figure which shows the function structural example which implements the speech recognition reliability calculation process of embodiment by this invention. The figure which shows the processing flow which implements the speech recognition reliability calculation process of embodiment by this invention. The figure which shows the specific example of the speech recognition reliability calculation process of embodiment by this invention. The figure which shows the dictionary structure using GMM and a silent model. The figure which shows the example of a recognition result and its phoneme reliability. Comparison table of recognition speed including reliability calculation. A comparison table of equal error rates for evaluating the degree of correctness / incorrectness discrimination ability.

Embodiments of the present invention will be described with reference to the drawings.
The speech unit likelihood ratio model creation device 1 according to the embodiment of the present invention is a device that performs speech recognition using the created speech unit likelihood ratio model, rather than being independently present (the present invention). In some cases, it is practical to exist as a component constituting the speech recognition reliability calculation device 2) according to the embodiment. Furthermore, the likelihood ratio model creation device 1 for each speech unit is not a component that constitutes the speech recognition reliability calculation device 2 so as to be easily separable from the speech recognition reliability calculation device 2, but is a speech recognition reliability calculation. It can also be said that the device 2 itself has been evaluated on one side with a certain function in mind. In short, it can be said that it is practically practical that the speech unit likelihood ratio model creation device 1 is the speech recognition reliability calculation device 2 itself.
However, the likelihood unit model creation device 1 for each speech unit exists as a single independent component, and the component that configures the speech recognition reliability calculation device 2 so as to be easily separable from the speech recognition reliability calculation device 2 It is not intended to exclude that. For example, if the purpose is to create a speech unit likelihood ratio model itself, there is no obstacle to realizing the speech unit likelihood ratio model creation device 1 as a single independent component.
Here, the speech recognition reliability calculation device 2 is realized by a computer such as a dedicated machine configured with dedicated hardware or a general-purpose machine such as a personal computer, and the likelihood ratio for each speech unit as an independent component. The same applies to the case where the model creation apparatus 1 is realized.

  A hardware configuration example in the case where the speech recognition reliability calculation device 2 is realized as a single component and realized by a computer will be described. The speech unit likelihood ratio model creation device 1 will be described as a constituent element of the speech recognition reliability calculation device 2.

<Hardware Configuration Example of Speech Recognition Reliability Calculation Device 2>
The speech recognition reliability calculation device 2 may include an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [cache memory, or the like. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. If necessary, the speech recognition reliability calculation device 2 may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the speech recognition reliability calculation device 2 stores a program for creating a likelihood ratio model for each speech unit, a program for calculating the speech recognition reliability, and data necessary for processing these programs. [Not limited to external storage devices, for example, a program may be stored in a ROM that is a read-only storage device. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  In the embodiment, the development data 200 is stored as data in a predetermined storage area of the storage unit. The development data 200 is composed of voice data (analog data of real voice) and correct answer labels based on voice units (for example, phonemes, syllables, semi-syllables, etc., which are phonemes in this embodiment) associated with the voice data. Contains multiple development data lists. However, the present invention is not limited to such a configuration. For example, it may be composed of an acoustic analysis result and a correct answer label in units of speech associated with the acoustic analysis result, or digitized speech data and It may be composed of correct labels in units of speech associated with the speech data. Such development data 200 may be the same as existing learning data used for acoustic model learning or the like. For example, the development data 200 preferably has a data amount exceeding 100 hours in terms of the total speech time length.

  Further, a GMM (Gaussian Mixture Model) 210 and a silent model 220 are stored as data in a predetermined storage area of the storage unit. For example, the GMM 210 is learned from all speech sections except for the unvoiced section of learning speech data (the learning speech data is not particularly limited) for learning an acoustic model used for speech recognition. It is a kind of acoustic model learned so as to include the characteristics of many phonemes (preferably all phonemes). The unvoiced model 220 is, for example, learning speech data for learning an acoustic model used for speech recognition (the learning speech data is not particularly limited, and is different from the learning speech data used for learning the GMM. This is a kind of acoustic model learned from a silent section of learning voice data).

  The storage unit of the speech recognition reliability calculation device 2 includes a program for calculating an acoustic feature, a program for calculating a correct likelihood, a program for calculating a GMM likelihood, a correct likelihood and a GMM likelihood. A program for calculating likelihood ratios by phoneme, a program for creating likelihood ratio models by phoneme, a program for performing speech recognition, a program for calculating reference likelihoods, and a recognition result likelihood And a program for calculating the likelihood ratio for each phoneme using the reference likelihood, a program for calculating the reliability for each phoneme, and a program for calculating the reliability for the recognition result.

In the speech recognition reliability calculation device 2, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and interpreted and executed by the CPU. As a result, the CPU performs predetermined functions (feature amount analysis unit, correct likelihood calculation unit, GMM likelihood calculation unit, first phoneme-specific likelihood ratio calculation unit, phoneme-specific likelihood ratio model generation unit, recognition processing unit, reference By implementing a likelihood calculation unit, a second phoneme-specific likelihood ratio calculation unit, a phoneme reliability calculation unit, and a recognition reliability calculation unit, creation of a phoneme-specific likelihood ratio model and calculation of speech recognition reliability are realized. The
The correct likelihood calculation unit, the GMM likelihood calculation unit, the first phoneme-specific likelihood ratio calculation unit, and the phoneme-specific likelihood ratio model creation unit are not essential components of the speech recognition reliability calculation device 2. In addition, the likelihood unit model creation device 1 for speech units according to the embodiment includes a feature amount analysis unit, a correct likelihood calculation unit, a GMM likelihood calculation unit, a first phoneme-specific likelihood ratio calculation unit, and a phoneme-specific likelihood ratio model. Although it is comprised including a preparation part, a recognition process part, a reference likelihood calculation part, a 2nd phoneme likelihood ratio calculation part, a phoneme reliability calculation part, a recognition reliability calculation part is a likelihood ratio model classified by speech unit It is not an essential component of the creation device 1.

  Next, as an embodiment, with reference to FIG. 2 to FIG. 8, the speech recognition reliability by the speech recognition reliability calculation device 2 including the speech unit likelihood ratio model creation processing by the speech unit likelihood ratio model creation device 1. The flow of the degree calculation process will be described narratively.

<Process for creating likelihood ratio model by phoneme>
The following processing is performed for each development data list (steps S1-S6).
The feature data analysis unit 101 collects a plurality of samples of the input audio signal (hereinafter referred to as a frame) using the audio data included in the development data list digitalized by an A / D conversion unit (not shown) as an input audio signal. For each, the same kind of acoustic feature quantity as that used for speech recognition processing such as cepstrum and speech power (for example, Reference 1) is calculated (step S2).
(Reference 1) Sadaaki Furui, “Digital Audio Processing”, Tokai University Press

Next, the correct likelihood calculating unit 102 uses the correct answer label corresponding to the input speech signal and the acoustic model 280 (phoneme environment dependent model, phoneme environment independent model, etc.) for the sound feature value for each frame. Recognize and detect the speech segment position of the phoneme included in the correct label (hereinafter referred to as phoneme alignment) A _correct (f, T) [f is the file name, T is the frame number] and the acoustic features of the frame T The correct likelihood P _correct (f, T) of the acoustic model 280 of the phoneme α (f, T) with respect to the quantity is calculated in each frame (step S3). The acoustic model 280 used here is the same as the acoustic model used during speech recognition.

A specific example will be described with reference to FIG. For example, speech data whose utterance content is “Tokyo” (the data name is 1.pcm for convenience) [frame length 29] and the phoneme string “t / ou / k / y / ou” of the correct label (The unit delimited by the symbol “/” is a phoneme), the correct likelihood calculating unit 102, for example, 5th to 7th utterance sections of the phoneme “t” included in the correct answer label The phoneme alignment A _correct ('1.pcm', 5 ≦ T ≦ 7) = “t” is detected for the frame, and the correct likelihood P of the acoustic model of the phoneme “t” with respect to the acoustic feature quantity of the fifth frame is also detected. _correct likelihood P in the order of _correct ('1.pcm', 5), correct likelihood P _correct of the acoustic model of phoneme "t" with respect to the acoustic feature of the 6th frame, etc. Find _correct ('1.pcm', T) [T = 5,6,7]. In addition, the correct likelihood calculating unit 102 is the utterance section of the phoneme “ou” in which the next 8th to 13th frames are included in the correct label, and therefore the phoneme alignment A _correct ('1.pcm') for the 8th to 13th frames. , 8 ≦ T ≦ 13) = “ou” is detected, and the correct likelihood P _correct ('1.pcm', 8) of the acoustic model of the phoneme “ou” with respect to the acoustic feature amount of each frame in the section ..., P _correct ('1.pcm', 13) is obtained. The same applies to the subsequent frames.
Note that “sil” in FIG. 4 is “silence”, that is, a silent section. Similarly, the phoneme alignment A _correct (f, T) and the correct likelihood P _correct (f, T) are also obtained for this unvoiced section.

Next, the GMM likelihood calculating unit 103 performs speech recognition using the acoustic feature amount for each frame, the GMM 210, and the unvoiced model 220, and calculates the GMM likelihood P _GMM (f, T) for the acoustic feature amount for each frame. (Step S4). The GMM 210 is a kind of acoustic model learned from all speech sections excluding the unvoiced section of the learning data. In actual speech recognition, the recognition target speech usually includes the unvoiced section. 103 performs speech recognition using the silent model 220 and obtains its likelihood. That is, the GMM likelihood calculation unit 103 obtains the likelihood for the acoustic feature amount by both models using a dictionary (grammar) composed of the GMM 210 and the unvoiced model 220 as shown in FIG. Degree P _GMM (f, T). In addition, the likelihood calculation itself by GMM should just use the method currently performed by the said nonpatent literature 2, etc., for example.

In the case of the specific example shown in FIG. 4, the GMM likelihood calculating unit 103 has GMM likelihoods P _GMM ('1.pcm', 1), P _GMM ('1. pcm ', 2), ..., P _GMM (' 1.pcm ', 29) is obtained.

Next, the first phoneme-specific likelihood ratio calculation unit 104 calculates the ratio P _correct (f, T) ÷ P _GMM (f, T) of the correct likelihood and the GMM likelihood obtained previously for each frame. It is calculated as a separate likelihood ratio P _ratio (f, T), and stored together with the phoneme alignment A _correct (f, T) in the phoneme-specific likelihood ratio storage unit 230 (step S5).
This process is performed for all frames of the input audio signal.

In the specific example shown in FIG. 4, for example, the likelihood ratio P _ratio ('1.pcm', 5) for the fifth frame (phoneme “t”) is “correct likelihood P _correct ('1.pcm'). , 5) "÷" GMM likelihood P _GMM ('1.pcm', 5) ". Also, the likelihood ratio P _ratio ('1.pcm', 8) for the 8th frame (phoneme “ou”) is “correct likelihood P _correct ('1.pcm', 8)” ÷ “GMM likelihood. Degree P _GMM ('1.pcm', 8) ". The same applies to other frames.

After each process of steps S2-S5 is performed on all the development data lists included in the development data 200 (step S6), the likelihood ratio model creation unit 105 for each phoneme creates a likelihood ratio model for each phoneme. (Steps S7 to S9). Specifically, the phoneme-specific likelihood ratio model creation unit 105 receives the phoneme-specific likelihood ratio P _ratio corresponding to the phoneme α for each phoneme type included in the development data 200 from the phoneme-specific likelihood ratio storage unit 230. Normalization that reads (f, T) and normalizes the probability distribution function (for example, normal distribution) with the likelihood ratio P _ratio (f, T) for each phoneme as a random variable by its maximum value (for example, cumulative value of appearance) The probability distribution function D (α) is stored in the phoneme-specific likelihood ratio model storage unit 240 as a phoneme-specific likelihood ratio model. This process is performed for all types of phonemes included in the development data 200. For example, when the development data 200 includes 30 types of phonemes such as “t” and “ou”, 30 types of likelihood ratio models for each phoneme such as D ('t') and D ('ou') exist. Will be created.
The above-described series of processing is <phoneme-specific likelihood ratio model creation processing>.

  According to the phoneme-specific likelihood ratio model created by the phoneme-specific likelihood ratio model creation processing, the phoneme-specific likelihood of the phoneme α is compared with the phoneme-specific likelihood ratio of the phoneme α obtained by the speech recognition processing described later. The output value of the frequency ratio model (the output value of the probability distribution function when the phoneme likelihood ratio is input as a random variable) is a high value with an upper limit of 1, and the output of the phoneme likelihood ratio model other than the phoneme α The value is a low value with 0 as the lower limit.

<Voice recognition reliability calculation processing>
Using the digital speech signal to be speech-recognized that has been digitized by an A / D conversion unit (not shown) as an input speech signal, the feature amount analysis unit 101 collects a plurality of samples (hereinafter referred to as a frame) of the input speech signal. In addition, an acoustic feature amount used for speech recognition processing such as cepstrum and speech power (see the above-mentioned Reference 1) is calculated (step S1p).

Next, the recognition processing unit 106 performs speech recognition processing using the acoustic feature value for each frame, the acoustic model 280, and a speech recognition dictionary (including a language model in some cases) 282, and recognizes the recognition result and the recognition result. Phoneme alignment A _result (T) of phonemes included in the frame and recognition result likelihood P _result (T) of each frame are obtained (step S2p). T represents a frame number.

A specific example will be described with reference to FIG. For example, when the first phoneme string of the recognition result for the input voice signal [frame length 27] whose utterance content is “Tokyo” is “t / ou / k / y / ou”, the recognition processing unit 106 For example, the phoneme alignment A _result (3 ≦ T ≦ 5) = “t” is obtained for the third to fifth frames, which are the utterance intervals of the recognition result phoneme “t”, and the recognition of the acoustic feature amount of each frame in the interval is obtained. The recognition result likelihood P _result (3),..., P _result (5) of the acoustic model of the result phoneme “t” is obtained. Similarly, the recognition processing unit 106 obtains phoneme alignment A _result (6 ≦ T ≦ 12) = “ou” for the sixth to twelfth frames that are the utterance interval of the next recognition result phoneme “ou”, and along with that, The recognition result likelihood P _result (6),..., P _result (12) of the acoustic model of the recognition result phoneme “ou” for the acoustic feature quantity of each frame in the section is obtained. The same applies to the subsequent frames.
Note that the phoneme alignment A _result (T) and the recognition result likelihood P _result (T) are similarly obtained for the unvoiced section sil in FIG.

Further, the reference likelihood calculating unit 107 performs speech recognition using the acoustic feature amount for each frame, the GMM 210, and the unvoiced model 220, and calculates the reference likelihood P _ref (T) for the acoustic feature amount for each frame (step) S3p). This process is the same as the process of the GMM likelihood calculating unit 103, and using the dictionary (grammar) composed of the GMM 210 and the unvoiced model 220 as shown in FIG. The larger one is defined as a reference likelihood P _ref (T). Note that the GMM 210 and the unvoiced model 220 used in the speech recognition reliability calculation process are the same as the GMM 210 and the unvoiced model 220 used in the phoneme-specific likelihood ratio model creation process.

In the case of the specific example shown in FIG. 7, the reference likelihood calculating unit 107 obtains reference likelihoods P _ref (1),..., P _ref (27) for the acoustic feature amount calculated from the input speech signal.

Next, the second phoneme-specific likelihood ratio calculation unit 108 obtains the ratio of the recognition result likelihood to the reference likelihood P _result (T) ÷ P _ref (T) for each frame, by the phoneme-specific likelihood ratio P _ratio (T ) And is stored in the phoneme-specific likelihood ratio storage unit 250 together with the phoneme alignment A _correct (f, T) (step S4p).

In the specific example shown in FIG. 7, for example, the likelihood ratio P _ratio (3) for each phoneme of the third frame (recognition result phoneme “t”) is “recognition result correct likelihood P _result (3)” ÷ “reference likelihood. Degree P _ref (3) ”. Further, the likelihood ratio P _ratio (6) for each phoneme of the sixth frame (recognition result phoneme “ou”) is calculated by “recognition result likelihood P _result (6)” ÷ “reference likelihood P _GMM (6)”. Is done. The same applies to other frames.

Subsequently, the phoneme reliability calculation unit 109 performs the following for phoneme alignment A _result (T _start (α) ≦ T ≦ T _end (α)) = “α” corresponding to one or more consecutive identical recognition result phonemes α. Processing is performed, and this processing is performed for all recognition result phonemes appearing in the recognition result (steps S5p-S7p). T _start (α) is the start frame number of the recognition result phoneme α, and T _end (α) is the end frame number of the recognition result phoneme α. In addition, even if the recognition result phonemes are the same type, the processing is applied individually when the start frame numbers are different. For example, in the example illustrated in FIG. 7, the recognition result phoneme “ou” appears twice in the recognition result (6th to 12th frames and 19th to 24th frames). Also, since the start frame numbers are different, the processing is applied individually, and T _start ('ou') = 6 and T _end ('ou') = 12, for the sixth to twelfth frames. T _start ('ou') = 19 and T _end ('ou') = 24 for the frame.
The details of the process are as follows. The phoneme reliability calculation unit 109 calculates each frame T = T _start (α), T _start (α) +1 within the phoneme alignment A _result (T _start (α) ≦ T ≦ T _end (α)) = “α”. ,..., T _end (α) −1, T _end (α) using the phoneme-specific likelihood ratio model D (α) of the phoneme α stored in the phoneme-specific likelihood ratio model storage unit 240. The output value D (α, P _ratio (T)) of the phoneme-specific likelihood ratio model D (α) corresponding to the phoneme-specific likelihood ratio P _ratio (T) stored in the phoneme-specific likelihood ratio storage unit 250. (Hereinafter referred to as frame reliability). Hereinafter, the frame reliability D (α, P _ratio (T)) (T _start (α) ≦ T ≦ T _end (α)) is changed to C _frame (T) (T _start (α) ≦ T ≦ T _end (α). ). The phoneme reliability calculation unit 109 then sets the frame reliability C _frame (T) (T _start (α) for this phoneme alignment A _result (T _start (α) ≦ T ≦ T _end (α)) = “α”. ≦ T ≦ T _end (α)) is divided by the number of frames (T _end (α) −T _start (α) +1) to obtain an average frame value A _result (T _start (α) ≦ The phoneme reliability C _phone [T _start (α): T _end (α)] when T ≦ T _end (α)) = “α” is set.

In the case of the specific example shown in FIG. 7, the second phoneme alignment from the beginning is A _result (3 ≦ T ≦ 5) = “t”, and therefore the phoneme “ phoneme-specific likelihood ratios P _ratio (3), P _ratio (4), and P _ratio (5) for each frame T using the phoneme-specific likelihood ratio model D ('t') of "t" The frame reliability C _frame (3), C _frame (4), and C _frame (5), which are values of the ratio model D ('t', P _ratio (T)), are obtained, and the average value of these frames is obtained to determine the phoneme confidence. Scale C _phone [3: 5]. Similar processing is performed for other phoneme alignments.

Next, the reliability calculation unit 110 calculates the average value of all phoneme alignments of the phoneme reliability C _phone calculated by the phoneme reliability calculation unit 109 as the reliability C of the recognition result (step S8p).

In the specific example illustrated in FIG. 7, an example of calculating the reliability C for the first recognition result is illustrated, but an embodiment in which the reliability is similarly obtained for each recognition result after the second recognition result is also permitted.
The series of processes described above is <voice recognition reliability calculation process>.

As for normalization of reliability, as shown in the first recognition result shown in FIG. 9, when the recognition result for the input speech signal is correct, the frame reliability C _frame (T) in any phoneme alignment is Since the value is a high value with 1 as the upper limit, the phoneme confidence measure C _phone and hence the reliability C are also high values.
On the other hand, if the recognition result for the input voice signal is incorrect, the input voice whose utterance content is “tyo (t / ou / k / y / ou)” as shown in the second recognition result in FIG. When the recognition result for the signal is “t / ou / ch / y / ou”, the phoneme “k” is erroneously recognized as the phoneme “ch”. In this phoneme alignment, it is estimated that a low frame reliability C _frame (T) is output from the phoneme-specific likelihood ratio model of the recognition result phoneme “ch” with respect to the phoneme feature amount originally uttered “k”. Is done. Therefore, the reliability of the entire recognition result also takes a low value compared to the reliability at the time of the correct answer, and the degree of confidence in the recognition result, that is, if the reliability is high, the probability of a correct answer is high. If the reliability is low, it can be expressed that the possibility of erroneous recognition is high.

≪Supplementary notes≫
In the embodiment, when only the creation of a speech unit likelihood ratio model is performed, each processing of steps S1p-S8p can be omitted. In the embodiment, when only the phoneme reliability is required, the process of step S8p can be omitted. Here, a phoneme is taken as an example of a speech unit. However, when other speech units such as a syllable are used, “phoneme” may be read as “syllable” or the like in the above description.

In a case where the speech unit likelihood ratio model creation device 1 and the speech recognition reliability calculation device 2 are configured as separate devices, for example, the speech unit likelihood ratio model creation unit 105 of the speech unit likelihood ratio model creation device 1 is used. Is stored in the storage unit of the speech recognition reliability calculation device 2 (for example, via a recording medium), and the phoneme reliability calculation unit 109 stores the stored likelihood ratio by speech unit. The phoneme reliability C _phone can be obtained using the model. In this case, the acoustic model 280 used by the speech recognition reliability calculation device 2 is preferably the same as the acoustic model 280 used by the speech unit likelihood ratio model creation device 1, but is not necessarily the same acoustic model. There is no need to use. As an example, the learning data used for learning the acoustic model is duplicated, but different acoustic models can be used when the structure of the acoustic model is different (such as the HMM (Hidden Markov Model) state or the number of mixtures). It is.

  In addition to the above-described embodiments, the speech unit likelihood ratio model creation device / method and the speech recognition reliability calculation device / method according to the present invention are not limited to the above-described embodiments, and depart from the spirit of the present invention. Changes can be made as appropriate without departing from the scope. In addition, the processing described in each embodiment may be executed not only in time series according to the description order, but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  Further, when the processing function in the speech unit-specific likelihood ratio model creation device / speech recognition reliability calculation device is realized by a computer, the function that the speech unit-specific likelihood ratio model creation device / speech recognition reliability calculation device should have The processing content of is described by a program. Then, by executing this program on a computer, the processing functions in the above-mentioned speech unit likelihood ratio model creation device / speech recognition reliability calculation device are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, the speech unit likelihood ratio model creation device / speech recognition reliability calculation device is configured by executing a predetermined program on a computer. However, at least a part of these processing contents is used. May be realized in hardware.

  Table 1 shown in FIG. 10 is a comparison table of recognition speed (RTF: Real Time Factor; a numerical value obtained by normalizing the recognition processing time by the voice length) including reliability calculation. Although the present invention uses a GMM, for example, even if a GMM having a Gaussian distribution of 64 mixtures is used in this embodiment, the calculation amount of this embodiment is far greater than the conventional method based on the likelihood of the syllable recognition result. Few. Rather, it can be seen from the comparison table that the present embodiment only requires the same amount of calculation as the conventional method based on N-best with a small amount of calculation.

  Table 2 shown in Fig. 11 shows the Equal Error Rate (EER: Equal Error Rate), which evaluates the level of correctness / error discrimination ability, and the false acceptance rate for falsely rejecting correct answers and falsely accepting errors outside the vocabulary. It is a comparison table in which it is better that the equal error rate is smaller with equal values. Since the present invention uses a GMM that includes a phoneme model, the present embodiment achieves an effect that is equal to or better than the conventional method based on the likelihood of syllable recognition results, and is more effective than the conventional method based on N-best. It can be seen that the equal error rate is low and the correct / incorrect discrimination ability is good.

Claims (9)

Storage means for storing an acoustic model, development data composed of speech data and correct labels associated with the speech data, and a mixed normal distribution (GMM);
Feature quantity analysis means for calculating the acoustic feature quantity of the voice data for each frame;
Correct likelihood calculation means for calculating the correct likelihood of phonemes included in the correct label using the correct label and the acoustic model for the acoustic feature amount for each frame;
GMM likelihood calculating means for calculating the likelihood by the GMM (GMM likelihood) for the acoustic feature amount for each frame;
For each frame, speech unit-specific likelihood ratio calculating means for calculating a ratio between the correct likelihood and the GMM likelihood as a first phoneme- specific likelihood ratio;
For each type of phonemes contained in the development data, phoneme corresponding said first phoneme likelihood normalize the probability distribution function of a random variable the ratio was normalized probability distribution function (phoneme likelihood ratio model) A speech unit-specific likelihood ratio model creation device including speech unit-specific likelihood ratio model creation means for creating The likelihood ratio model creation device for each voice unit according to claim 1,
The GMM is a likelihood ratio model creation device by speech unit, characterized in that it is a mixed normal distribution learned from voiced sections of speech data for learning. In the speech unit likelihood ratio model creation device according to claim 1 or 2,
The speech unit likelihood ratio model creating apparatus characterized in that the GMM likelihood calculating means calculates the GMM likelihood using an unvoiced model learned from an unvoiced section of learning speech data. The storage unit stores an acoustic model, development data including voice data and a correct answer label associated with the voice data, and a mixed normal distribution (GMM).
A feature amount analyzing step for calculating an acoustic feature amount of the voice data for each frame;
A correct likelihood calculation step of calculating a correct likelihood of a phoneme included in the correct label using the correct label and the acoustic model for the acoustic feature amount for each frame;
A GMM likelihood calculating step for calculating a likelihood by the GMM (GMM likelihood) for the acoustic feature amount for each frame;
For each frame, and the ratio of the correct answers likelihood and the GMM likelihood first Ruoto voice unit by the likelihood ratio calculation step to calculate a phoneme likelihood ratio,
For each type of phonemes contained in the development data, phoneme corresponding said first phoneme likelihood normalize the probability distribution function of a random variable the ratio was normalized probability distribution function (phoneme likelihood ratio model) A method of creating a likelihood ratio model for each speech unit, comprising a step of creating a likelihood ratio model for each speech unit for creating a speech unit. Storage means for storing an acoustic model, a likelihood ratio model by phoneme created by the likelihood ratio model creation apparatus by speech unit according to any one of claims 1 to 3, and a mixed normal distribution (GMM) When,
Feature quantity analysis means for calculating the acoustic feature quantity of the speech signal to be recognized for each frame;
Recognition processing means for calculating a speech recognition result and a recognition result likelihood of a phoneme included in the speech recognition result using the acoustic model for the acoustic feature amount for each frame;
Reference likelihood calculating means for calculating a likelihood (reference likelihood) by the GMM for the acoustic feature amount for each frame;
For each frame, speech unit-specific likelihood ratio calculating means for calculating a ratio between the recognition result likelihood and the reference likelihood as a second phoneme- specific likelihood ratio;
For each phoneme contained in the speech recognition result, the output value of the phoneme likelihood ratio model when the inputs the second phoneme likelihood ratio for each frame corresponding to the phoneme (frame reliability) A speech recognition reliability calculation device including speech unit reliability calculation means for determining each frame reliability and obtaining a frame average value of the frame reliability as a speech unit reliability. In the speech recognition reliability calculation apparatus according to claim 5,
A speech recognition reliability calculation device comprising: a reliability calculation means for calculating an average value of the speech unit reliability corresponding to each phoneme included in the speech recognition result as a reliability of the speech recognition result. In the speech recognition reliability calculation apparatus according to claim 5 or 6,
The acoustic model is the same acoustic model as the acoustic model used in creating the phoneme likelihood ratio model,
The speech recognition reliability calculation apparatus according to claim 1, wherein the GMM is the same GMM as the GMM used when creating the likelihood ratio model for each phoneme . The storage means includes an acoustic model, a likelihood ratio model by phoneme created by the speech unit likelihood ratio model creation device according to any one of claims 1 to 3, a mixed normal distribution (GMM), and Is remembered,
A feature amount analyzing step for calculating an acoustic feature amount of a speech signal to be recognized for each frame;
A recognition processing step of calculating a speech recognition result and a phoneme recognition result likelihood included in the speech recognition result using the acoustic model for the acoustic feature amount for each frame;
A reference likelihood calculating step for calculating a likelihood (reference likelihood) by the GMM with respect to the acoustic feature amount for each frame;
For each frame, a speech unit-specific likelihood ratio calculation step of calculating a ratio between the recognition result likelihood and the reference likelihood as a second phoneme- specific likelihood ratio;
For each phoneme contained in the speech recognition result, the output value of the phoneme likelihood ratio model when the inputs the second phoneme likelihood ratio for each frame corresponding to the phoneme (frame reliability) A speech recognition reliability calculation method including: a speech unit reliability calculation step that calculates each frame average value of the frame reliability as a speech unit reliability.       A computer functions as the speech unit likelihood ratio model creation device according to any one of claims 1 to 3 and / or the speech recognition reliability calculation device according to any one of claims 5 to 7. Program to let you.

Images