JP5749186B2 - Acoustic model adaptation device, speech recognition device, method and program thereof - Google Patents

Description

  The present invention relates to an acoustic model adaptation device that performs unsupervised acoustic model adaptation, a speech recognition device that uses the acoustic model, a method thereof, and a program.

  When updating the acoustic model used for speech recognition, model parameter optimization processing is performed so that as many examples as possible are included in the learning data. This process is called “acoustic model adaptation” and generally uses a speech file and a correct text representing the utterance content of the speech file as learning (adaptive) data. The adaptation of the acoustic model is broadly classified into supervised adaptation in which correct text is obtained by human writing up the reading corresponding to the speech file, and unsupervised adaptation obtained as a speech recognition result of the speech file. Unsupervised adaptation can reduce the development cost of a speech recognition system because it does not require manual correct text.

  For example, in Non-Patent Document 1, when performing unsupervised acoustic model adaptation, the speech accumulated in the system is classified into several classes based on the acoustic similarity, and unsupervised acoustic model adaptation is applied to each class. By generating an acoustic model for each class, and when speech is input to the system, an appropriate class acoustic model is selected and used for speech recognition, resulting in better speech recognition accuracy than normal unsupervised acoustic model adaptation. Enhanced techniques are disclosed.

  The technology learns the speech GMM (Gaussian Mixture Model) for each class of the accumulated speech at random, and then reassigns each speech to the class that maximizes the GMM likelihood. This is a method of classifying into classes and applying the unsupervised acoustic model for each class. When speech recognition is performed, the likelihood for the input speech is calculated by the speech GMM of each class, and the acoustic model of the class having the maximum likelihood is selected to perform speech recognition.

Sato et al. "Acoustic model adaptation by selective learning based on two-stage clustering" IEICE Transactions, D-II, Vol. J85-D-II, No2, pp.174-183, 2002.

  In the conventional method disclosed in Non-Patent Document 1, the likelihood calculated by the speech GMM is used as the similarity between speech. In speech GMM learning, all phonemes are identified without considering the type of phoneme uttered in the speech, and one speech GMM parameter is determined. For this reason, if the distance between speech GMMs is used as the similarity between speeches, the similarity between two phonemes that differ in terms of phonemes will be estimated high, and greatly different phonemes may be classified into the same class. is there. Even if acoustic model adaptation is applied to such a class, it is difficult to set parameters so that different large phonemes can be recognized correctly (the other cannot be recognized if it is matched), so the effect of adaptation Will become smaller.

  The present invention has been made in view of such a problem, and is an acoustic model adaptation that performs unsupervised acoustic model adaptation by evaluating similarity for each phoneme and classifying voices with high similarity into the same class. It is an object to provide a device, a speech recognition device, and a method and program thereof.

  The acoustic model adaptation device of the present invention includes a speech recognition unit, a phoneme error tendency vector generation unit, a clustering unit, a base acoustic model, a base acoustic model adaptation unit, and a post-adaptation acoustic model recording unit. The voice recognition unit receives a voice group composed of a plurality of voices, and outputs a voice recognition result text obtained as a result of voice recognition processing of the input voice based on the base acoustic model and the voice. The phoneme error tendency vector generation unit performs acoustic feature extraction of the input speech for each frame, obtains the output probability of the acoustic feature of the frame for all states of all phonemes included in the base acoustic model, and calculates the maximum value of the output probability Is divided by the sum of the output probabilities of the corresponding frame to obtain the posterior probability of the first phoneme, and the posterior probability obtained by arranging the phoneme posterior probabilities obtained by calculating the average value of each first phoneme posterior probability in units of speech. The speech group overall posterior probability vector of the entire speech group obtained in advance is subtracted from the posterior probability vector as a vector to generate the phoneme error tendency vector of the speech. The clustering unit receives the three groups of the speech, the speech recognition result text, and the phoneme error tendency vector as input, and classifies the three groups into clusters of a predetermined class on the basis of the similarity between the phoneme error tendency vectors. In addition, a centroid that is an average vector of phoneme error tendency vectors in the cluster is obtained, and the cluster and the centroid are output. The base acoustic model adaptation unit generates a post-adaptation acoustic model in which the base acoustic model is adapted for each cluster, based on the speech and speech recognition results included in each cluster, with the cluster and centroid as inputs. The post-adaptation acoustic model recording unit records the post-adaptation acoustic model for each cluster.

  The speech recognition apparatus of the present invention further includes an after-adaptation acoustic model recording unit, a phoneme error tendency vector generation unit, a neighborhood centroid selection unit, an applied acoustic model, and a speech recognition unit. The post-adaptation acoustic model recording unit records a plurality of clusters including the post-adaptation acoustic model generated by the above-described acoustic model adaptation device and their centroids. The phoneme error tendency vector generation unit performs acoustic feature extraction of the recognition target speech for each frame, obtains the output probability of the acoustic feature of the frame for all states of all phonemes included in the base acoustic model, and calculates the maximum of the output probability The value is divided by the sum of the output probabilities of the frame to obtain the posterior probability of the first phoneme, and the posterior probabilities obtained by calculating the average value for each phoneme of the posterior probability of the first phoneme in units of speech A probability vector is generated as a phoneme error tendency vector at the time of recognition by subtracting the entire speech group a posteriori probability vector of the entire speech group input in advance from the posterior probability vector. The nearest neighbor centroid selection unit selects the post-adaptation acoustic model that maximizes the similarity between the recognition-time phoneme error tendency vector and the centroid of a plurality of clusters and pairs, and outputs the selected acoustic model as an applied acoustic model. The speech recognition unit performs speech recognition processing on the recognition target speech based on the applied acoustic model, and outputs a speech recognition result text.

  The acoustic model adaptation device according to the present invention obtains a phoneme-number-dimensional phoneme error tendency vector in consideration of a tendency for each phoneme for each voice, and each voice and its speech recognition result text based on the similarity between the phoneme error tendency vectors. Are classified into a plurality of clusters. Therefore, voices having similar phoneme error tendencies are classified into each cluster. Using the speech with similar phoneme error tendency and the speech recognition result text, the base acoustic model is adapted unsupervised, so that the adaptation effect of the acoustic model can be enhanced.

  Further, according to the speech recognition apparatus of the present invention, the phoneme error tendency vector at the time of recognition of the recognition target speech is obtained, and the post-adaptive sound adapted by the above-described acoustic model adaptation device of the present invention and the recognized phoneme error tendency vector. Since the speech recognition process is performed using the applied acoustic model in which the post-adaptation acoustic model having a high similarity with the model centroid (phoneme error tendency vector) is selected, the recognition accuracy of speech recognition can be improved.

The figure which shows the function structural example of the acoustic model adaptation apparatus 100 of this invention. The figure which shows the operation | movement flow of the acoustic model adaptation apparatus 100. The figure which shows the more specific functional structural example of the phoneme error tendency vector generation part 20. FIG. The figure which shows typically the relationship between the acoustic feature-value _Oit and output probability for every flame | frame. The figure which shows typically the relationship between the posterior probability of a 1st phoneme, and the posterior probability for every phoneme. The figure which shows the function structural example of the speech recognition apparatus 200 of this invention. The figure which shows the operation | movement flow of the speech recognition apparatus 200. The figure which shows an evaluation experiment result.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows a functional configuration example of an acoustic model adaptation apparatus 100 of the present invention. The operation flow is shown in FIG. The acoustic model adaptation apparatus 100 includes a speech recognition unit 10, a phoneme error tendency vector generation unit 20, a clustering unit 30, a base acoustic model 40, a base acoustic model adaptation unit 50, a post-adaptation acoustic model recording unit 60, It comprises. The acoustic model adaptation apparatus 100 is realized by a predetermined program being read into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The voice recognition unit 10 receives a voice group consisting of a plurality of voices, and outputs a voice recognition result text and a voice as a result of voice recognition processing of the voice based on the base acoustic model 40 (step S10). Voice is given as a digital signal, and voice recognition processing is performed for each frame in which a predetermined number of voice digital signals are one frame (for example, 20 ms). The speech recognition result text resulting from the speech recognition is output in combination with the speech. Note that the voice recognition unit 10 is configured by a conventional technique.

  The phoneme error tendency vector generation unit 20 extracts the acoustic feature amount of the speech digital signal for each frame, obtains the output probability of the acoustic feature amount of the frame for all states of all phonemes included in the base acoustic model 40, and outputs The maximum probability value is divided by the sum of the output probabilities of the frame to obtain the posterior probability of the first phoneme, and the phoneme posterior probability obtained by calculating the average value for each phoneme of the posterior probability of the first phoneme in the speech unit. Are set as posterior probability vectors, and the speech group posterior probability vector of the entire speech group obtained in advance is subtracted from the posterior probability vector to generate a phoneme error tendency vector for the speech unit (step S20). Here, each of the plurality of voices constituting the voice group is, for example, a voice recorded in a telephone conversation for several minutes or a lecture for about one hour, a voice of a moving picture, or the like. Here, the entire speech group posterior probability vector of the entire speech group is a process of generating a posterior probability vector for the input speech group in order to create an acoustic model performed by the phoneme error tendency vector generation unit 20. This is a posterior probability vector generated in advance and is obtained in advance. The entire speech group posterior probability vector may be input from the outside, or may be recorded in advance in the phoneme error tendency vector generation unit 20. A detailed operation description of the phoneme error tendency vector generation unit 20 including the entire speech group posterior probability vector will be described later.

  The clustering unit 30 is an entire set that is a set of three sets of the speech recognition result text output from the speech recognition unit 10 and the speech thereof, and the phoneme error tendency vector of the speech output from the phoneme error tendency vector generation unit 20. Is an average vector of phoneme error tendency vectors in the cluster, classifying the three groups into clusters of a predetermined number (k) classes using the similarity between phoneme error tendency vectors as a scale. The centroid is obtained, and the cluster and centroid are output (step S30). One cluster is a subset of three sets of speech, speech recognition result text, and phoneme error tendency vector. The classification number k is a number that divides the entire set into a desired number of clusters. For example, k = 48. The classification number k may be given to the clustering unit 30 from the outside, or may be set in the clustering unit 30 in advance. The classification number k is set from 50 to 100 as a guide.

The base acoustic model adaptation unit 50 receives the clusters and centroids output from the clustering unit 30 as input, and adapts the base acoustic model 40 for each cluster based on the speech and speech recognition result text included in each cluster. An acoustic model is generated, and an after-adaptation acoustic model and a centroid pair of the cluster are output (step S50). The base acoustic model adaptation unit 50 includes k acoustic model adaptation units 50 ₁ to 50 _k that respectively correspond to several k clusters.

The post-adaptation acoustic model recording unit 60 records a post-adaptation acoustic model and centroid pair for each cluster (step S60). The plurality of after-adaptation acoustic models _{1 to k} are separately recorded as after-adaptation acoustic models 60 _{1 to} 60 _k .

  By operating as described above, the base acoustic model 40 is subjected to an unsupervised acoustic model adaptation process using the speech and speech recognition result text for each speech subset (cluster) having a similar phoneme error tendency. Converted to an acoustic model. Therefore, the acoustic model adaptation apparatus 100 can enhance the adaptation effect of the acoustic model as compared with the conventional acoustic model adaptation apparatus.

[Phoneme error tendency vector generator]
The function of each part of the acoustic model adaptation apparatus 100 will be specifically described and described in detail. FIG. 3 shows a functional configuration example of the phoneme error tendency vector generation unit 20. The phoneme error tendency vector generation unit 20 includes an acoustic feature quantity extraction unit 21, an output probability calculation unit 22, an output probability sum calculation unit 23, a maximum output probability phoneme posterior probability calculation unit 24, and a phoneme posterior probability calculation unit 25. And a posteriori probability vector generation means 26 and a phoneme error tendency vector generation means 27.

The acoustic feature quantity extracting unit 21, a plurality of audio _{U all (U 1, U 2} , ..., U n) for extracting acoustic features O _it voice _{U i} for each frame. t represents a frame number in the voice U _i . The acoustic feature quantity O _it is calculated by, for example, Mel Frequency Cepstral Coefficients (MFCC) analysis. The acoustic feature amount _Oit may be obtained from the speech recognition unit 10. In that case, the acoustic feature quantity extraction means 21 is unnecessary.

Output probability calculation unit 22, in each frame, for all states of all phonemes monophone models included in the base acoustic model 40, and calculates the output probability of the acoustic feature quantity O _it of the frame. The output probability of the acoustic feature quantity _Oit of the t-th frame is expressed as P ( _Oit | _sj ). Here, s _j represents the j-th state of the monophone HMM included in the base acoustic model 40.

FIG. 4 schematically shows the relationship between the acoustic feature amount _Oit and the output probability for each frame. The acoustic feature value _Oit is extracted for each frame having a time width of 20 ms, for example, from the head of the sound U _i . The output probability of the acoustic feature value O _i8 of the eighth frame t _i8 is calculated for all states of all phoneme monophone models included in the base acoustic model 40. The total phonemes are phonemes corresponding to the Roman alphabet notation of / a /, / i /, / u /,... / N /, for example, and the number thereof is 26, for example. The total state of the phoneme monophone model means all states from the first state to the third state of / a / to / N /, and 78 if the number of phonemes is 26 and the number of states is 3. become. The output probability calculation means 22 calculates the output probabilities P (O _i8 | a ₁ ) to P (O _i8 | N ₃ ) for all the states. P (O _i8 | a ₁ ) to P (O _i8 | N ₃ ) are P (O _i8 | s ₁ ) to P (O _i8 | s ₇₈ ).

The output probability sum calculating means 23 calculates the sum P (O _it ) of the output probabilities of each frame by the equation (1).

The a posteriori probability calculating means 24 of the maximum output probability phoneme takes the maximum output probability (formula (2)) as the sum of the output probabilities P of each frame, where ^ p is the phoneme having the maximum output probability in that frame. The value divided by (O _it ) is calculated as the posterior probability P (^ p | O _it ) of the first phoneme of the frame (formula (3)). FIG. 4 shows an example in which the second state of phonemes / i / is set as the maximum output probability.

  The numerator of equation (3) may be an output probability in a state indicating the maximum output probability, or may be a sum of output probabilities of the first state to the third state of the phoneme.

The phoneme posterior probability calculation means 25 obtains the frame average value for each phoneme of the posterior probability of the first phoneme of each frame in units of speech U _{i and} sets it as the posterior probability for each phoneme. FIG. 5 schematically shows the relationship between the posterior probability of the first phoneme and the posterior probability of each phoneme. FIG. 5 shows a case _where the first phoneme of frames t ₈ , t ₂₄ and t _n in speech U _i is / i /. The notation of the posterior probability of the first phoneme of other phonemes is omitted.

The phoneme posterior probability calculating means 25 calculates the sum of the posterior probabilities of the frame with the first posterior probability of the phoneme / i / over all the frames in the speech U _i , and the number of frames with the first posterior probability of the phoneme / i /. Divide by to calculate the phoneme posterior probability for each phoneme. The phoneme posterior probabilities for each other phoneme are calculated similarly.

The posterior probability vector generating means 26 generates a posterior probability vector by arranging phoneme posterior probabilities for each phoneme. The order in which phoneme posterior probabilities are arranged may be, for example, an order in which phoneme names are arranged in ascending order in a lexicographic order. The posterior probability vector is assumed to be sparse because there are few phonemes that appear when the time length of the speech U _i is short. However, as described above, since the voice U _i uses a long unit such as one call in a telephone conversation as one adaptation data, the phoneme posterior probabilities of all phonemes are often arranged. For example, when the number of phonemes is 26, the posterior probability vector is a 26-dimensional vector.

The phoneme error tendency vector generation means 27 subtracts the previously obtained speech group overall posterior probability vector input from the outside from the posterior probability vector output from the posterior probability vector generation means 26 to thereby obtain the phoneme error tendency of each voice U _i. Generate a vector. The entire speech group posterior probability vector is generated by performing processing until the posterior probability vector generated by the phoneme error tendency vector generation unit 20 is generated for all speech U _i input to the acoustic model adaptation apparatus 100. . By making the base acoustic model 40 the same, the number of dimensions of the posterior probability vector and the entire speech group posterior probability vector are the same.

The phoneme error tendency vector generation means 27 performs a process of normalizing the posterior probability of each phoneme of each speech U _i with the posterior probability of all speech for creating an acoustic model. The value of each element of the phoneme error tendency vector obtained by this processing is a phoneme error tendency score representing the recognition error tendency of the phoneme. In a certain voice U _i , when the error tendency score value of a phoneme is positive, it indicates that the phoneme can be recognized more correctly than usual. A negative value indicates that the phoneme is more likely to be erroneously recognized than usual.

[Clustering section]
The clustering unit 30 receives as input three sets of speech recognition result text and speech U _i output from the speech recognition unit 10 and phoneme error tendency vectors of the speech U _i output from the phoneme error tendency vector generation unit 20. , Using the similarity between phoneme error tendency vectors as a scale, classify the three groups into clusters of a predetermined number (k) classes, and a centroid that is an average vector of phoneme error tendency vectors in the clusters; And the cluster and centroid are output.

The k clusters are classified based on the similarity between vectors. As a measure of similarity between vectors, for example, cosine similarity can be used. The cosine similarity S (V ₁ , V ₂ ) between _two vectors V ₁ and V ₂ having the same number of dimensions is calculated by Expression (4).

V ₁ · V ₂ represents the inner product of the vectors V ₁ and V ₂ , and | V ₁ | and | V ₂ | represent the norms of the vectors V ₁ and V ₂ , respectively.

The clustering unit 30 calculates the cosine similarity between the respective voices by using the phoneme error tendency vectors of the respective voices U _i as V ₁ and V ₂ , and uses the cosine similarity values for the respective voices. Classify U _i into k clusters. Then, the phoneme error tendency vectors in each cluster are averaged to obtain the centroid of the cluster. For the process of classifying a plurality of voices U _i into k clusters, for example, a known k-means method that can process relatively large speed even with large-scale data can be used. Since the classification process itself is not the main part of the present invention, detailed description thereof is omitted.

[Base acoustic model adaptation section]
The base acoustic model adaptation unit 50 includes a number of acoustic model adaptation units 50 ₁ to 50 _k corresponding to the number k of clusters. The acoustic model adaptation units 50 ₁ to 50 _{k each} receive _k clusters composed of a subset of three groups of speech, speech recognition result text, and phoneme error tendency vector, and speech included in each cluster. Based on U _i and the speech recognition result text, the base acoustic model 40 is adapted without supervision for each cluster to generate an after-adaptation acoustic model, and a pair of the after-adaptation acoustic model and the centroid of the cluster is output.

  The acoustic model adaptation algorithm is a conventional technique, for example, J.-L. Gauvain and C.-H. Lee, “Maximum a Posteriori Estimation for Multivariate Gaussian Mixture Observations of Markov Chains,” IEEE trans. On Speech and Audio. processing, 2 (2), pp.291-298, 1994).

  The post-adaptation acoustic model in which the base acoustic model 40 is adapted without supervision for each cluster is recorded in the post-adaptation acoustic model recording unit 60 for each cluster in combination (pair) with the centroid. Since the post-adaptation acoustic model is obtained by performing unsupervised acoustic model adaptation for each speech unit having a high degree of similarity, the acoustic model having a high adaptation effect can be obtained.

  Next, a speech recognition device 200 that performs speech recognition processing using the post-adaptation acoustic model generated by the acoustic model adaptation device 100 of the present invention will be described.

[Voice recognition device]
FIG. 6 shows a functional configuration example of the speech recognition apparatus 200 of the present invention. The operation flow is shown in FIG. The speech recognition apparatus 200 includes an after-adaptation acoustic model recording unit 60, a recognition phoneme error tendency vector generation unit 210, a base acoustic model 40, a nearest centroid selection unit 220, an applied acoustic model 230, and a speech recognition unit. 240. The post-adaptation acoustic model recording unit 60 and the base acoustic model 40 are the same as the acoustic model adaptation apparatus 100 as apparent from the reference numerals. The acoustic model adaptation apparatus 100 is realized by a predetermined program being read into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The post-adaptation acoustic model recording unit 60 records a set of a plurality of clusters in which the post-adaptation acoustic model generated by the acoustic model adaptation apparatus 100 is recorded and its centroid.

  The recognition-time phoneme error tendency vector generation unit 210 performs acoustic feature extraction of the recognition target speech for each frame, obtains an output probability of the acoustic feature of the frame for all states of all phonemes included in the base acoustic model, and outputs the output The maximum probability value is divided by the sum of the output probabilities of the frame to obtain the posterior probability of the first phoneme, and the phoneme posterior probability obtained by calculating the average value for each phoneme of the posterior probability of the first phoneme in the above speech unit By arranging the units as a posterior probability vector, the utterance group posterior probability vector of the entire speech group obtained in advance is subtracted from the posterior probability vector to generate the phoneme error tendency vector during speech recognition (step S210). The processing performed by the recognition phoneme error tendency vector generation unit 210 is the same as the processing performed by the phoneme error tendency vector generation unit 20 of the acoustic model adaptation device 100. The only difference is that the recognition target speech to be recognized is input to the recognition-target phoneme error tendency vector generation unit 210.

  The nearest-neighbor centroid selection unit 220 selects an after-adaptation acoustic model in which the similarity between a recognition phoneme error tendency vector and a plurality of clusters and a set of centroids recorded in the after-adaptation acoustic model recording unit 60 is maximum. It selects and outputs as an applicable acoustic model (step S220). A centroid paired with a plurality of clusters recorded in the post-adaptation acoustic model recording unit 60 is an average of phoneme error tendency vectors in each cluster. The nearest-neighbor centroid selection unit 220 uses the cosine similarity (the same as that used in the clustering unit 30 of the acoustic model adaptation device 100) by using the similarity between the centroid (phoneme error tendency vector) and the recognition phoneme error tendency vector. Evaluation is performed using Expression (4), and the post-adaptation acoustic model that maximizes the similarity is selected and output as the applied acoustic model 230.

  The speech recognition unit 240 performs speech recognition processing on the recognition target speech based on the applied acoustic model 230 selected by the nearest centroid selection unit 220, and outputs a speech recognition result text (step S240).

  According to the speech recognition apparatus 200, since the speech recognition process is performed using an acoustic model adapted to speech with a similar phoneme error tendency to the recognition target speech, the recognition rate can be improved. Further, in the conventional method, the likelihood for speech is calculated for each of a plurality of acoustic models and an appropriate acoustic model is selected, whereas in the method of the present invention, one piece obtained from one base acoustic model is used. By generating the phoneme error tendency vector at the time of recognition, the phoneme error tendency of each voice is evaluated, so that an appropriate acoustic model can be selected at high speed. As a result, it is possible to reduce the time lag from when the voice is input until the voice recognition result is output.

[Evaluation experiment]
An evaluation experiment was conducted for the purpose of confirming the effects of the acoustic model adaptation method and the speech recognition method of the present invention. The result will be described.

  First, experimental conditions will be described. As voice data for the evaluation experiment, a telephone conversation voice 391 call by 24 speakers was used. Twelve to twenty-two calls were recorded per person. Create a data set that collects half of each speaker's call (196 calls) and a data set that collects the remaining calls (196 calls). When recognizing one data set, the other data set It was used as adaptive data and evaluated using the average recognition rate of both experiments.

  The k-means method was used for clustering of the adaptive data, and the number of clusters was 48. MAP adaptation (reference: “Gauvain et al., IEEE Trans. SAP, 2 (2), 291-298, 1994”) was used for unsupervised adaptation of acoustic models.

  Based on the above assumptions, the recognition rates of the following four conditions were compared. The result is shown in FIG. The horizontal axis is the speaker ID, and the vertical axis is the recognition rate. The notation in FIG. 8, ■ indicates a recognition rate (baseline) using the base acoustic model as it is, and × indicates a recognition rate using one adaptive model (one cluster) created using all adaptive data without performing clustering. ), ● are recognition rates using selected clustered acoustic models (this invention), ▲ are recognition rates using acoustic models that have been adapted by manually classifying adaptation data for each speaker (speaker known) Condition).

  Compared with one cluster (×), this invention (●) improved the recognition rate for 23 speakers out of 24. Since the recognition rates of the remaining speakers (ID = 12) are almost the same, the effectiveness of clustering and model selection focusing on the phoneme error tendency of the present invention was confirmed. Further, as shown in FIG. 8, the recognition rate (●) of the present invention is a recognition rate close to the speaker known condition (▲) considered to be an ideal condition. The average recognition rates of the 24 speakers in the present invention and the known speaker conditions are 84.55% and 84.50%, respectively, and it is confirmed that the recognition performance equivalent to the known speaker conditions can be obtained by the method of the present invention. did it.

  As described above, the adaptive data clustering is performed by clustering the adaptive data by paying attention to the similarity of the phoneme error tendency based on the speech data, and generating the post-adaptation acoustic model by applying the base acoustic model unsupervised for each classified cluster. The speech recognition apparatus 200 that selects and uses the post-adaptation acoustic model can improve the speech recognition rate.

  The processes described in the above apparatuses and methods are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. .

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (6)

A voice recognition unit that outputs a voice recognition result text and a voice as a result of voice recognition processing based on a base acoustic model, with a voice group including a plurality of voices as an input;
The acoustic feature extraction of the voice is performed for each frame, and the output probability of the acoustic feature of the frame is obtained for all states of all phonemes included in the base acoustic model, and the maximum value of the output probability is calculated as the output probability of the frame. Divide by the sum to make the posterior probability of the first phoneme, arrange the phoneme posterior probabilities obtained by the above speech units for the average value for each phoneme of the posterior probability of the first phoneme as the posterior probability vector A phoneme error tendency vector generation unit that subtracts a speech group posterior probability vector of the entire speech group obtained in advance to generate a phoneme error tendency vector of the speech;
Three sets of the speech, the speech recognition result text, and the phoneme error tendency vector are input, and the three sets are clustered in a predetermined number of classes using the similarity between the phoneme error tendency vectors as a scale. A centroid that is an average vector of the phoneme error tendency vectors in the cluster and outputs the cluster and the centroid;
Based on the cluster and centroid as an input, based on the speech and speech recognition result text included in each cluster, a base acoustic model adaptation unit that generates an adaptive acoustic model in which the base acoustic model is adapted for each cluster;
A post-adaptation acoustic model recording unit that records the post-adaptation acoustic model for each cluster;
An acoustic model adaptation apparatus comprising: The acoustic model adaptation device according to claim 1,
The phoneme error tendency vector generation unit
An acoustic feature quantity extraction means for extracting the acoustic feature quantity for each frame using the voice as input and outputting the acoustic feature quantity;
Output probability calculation means for calculating the output probability of the acoustic feature amount for each frame for all states of all monophone models of the base acoustic model;
Output probability sum calculation means for calculating the sum of output probabilities calculated for all the monophone models for each frame;
The maximum output probability of each first frame is defined as the maximum output probability of the first phoneme, and the maximum output probability of the first phoneme is calculated by dividing the maximum output probability of the first phoneme by the sum of the output probabilities of all the states. A means for calculating the posterior probability of a stochastic phoneme;
A phoneme posterior probability calculating means for calculating a phoneme posterior probability for each phoneme by dividing a value obtained by summing the posterior probability of the first phoneme over all frames by a number when the phoneme is the first phoneme;
A posterior probability vector generating means for generating the posterior probability vector by arranging the phoneme posterior probabilities in units of the speech;
A phoneme error tendency vector generation means for subtracting a speech group posterior probability vector of the whole speech group obtained in advance from the outside, and generating the phoneme error trend vector of the speech as the posterior probability vector;
An acoustic model adaptation device comprising: A post-adaptation acoustic model recording unit that records a plurality of clusters including the post-adaptation acoustic model generated by the acoustic model adaptation device according to claim 1 and the centroid, and
The acoustic feature extraction of the recognition target speech is performed for each frame, the output probability of the acoustic feature of the frame is obtained for all the states of all phonemes included in the base acoustic model, and the maximum value of the output probability is calculated as the output probability of the frame. Divide by the sum to make the posterior probability of the first phoneme, arrange the phoneme posterior probabilities obtained by the above speech units for the average value for each phoneme of the posterior probability of the first phoneme as the posterior probability vector A recognition-time phoneme error tendency vector generation unit that generates a recognition-time phoneme error tendency vector by subtracting the entire speech group posterior probability vector of the entire speech group that is input in advance from the outside;
The nearest-neighbor centroid selection unit that selects and outputs the applied acoustic model as an applied acoustic model that maximizes the degree of similarity between the recognition phoneme error tendency vector and the centroid of the set of the plurality of clusters,
A speech recognition unit that performs speech recognition processing on the recognition target speech based on the applied acoustic model and outputs a speech recognition result text; and
A speech recognition apparatus comprising: A speech recognition process for outputting a speech recognition result text and a speech as a result of speech recognition processing based on a base acoustic model, with a speech group consisting of a plurality of speeches as input,
The acoustic feature extraction of the voice is performed for each frame, and the output probability of the acoustic feature of the frame is obtained for all states of all phonemes included in the base acoustic model, and the maximum value of the output probability is calculated as the output probability of the frame. Divide by the sum to make the posterior probability of the first phoneme, arrange the phoneme posterior probabilities obtained by the above speech units for the average value for each phoneme of the posterior probability of the first phoneme as the posterior probability vector A phoneme error tendency vector generation process for subtracting a speech group posterior probability vector of the entire speech group obtained in advance to generate a phoneme error tendency vector of the speech;
Three sets of the speech, the speech recognition result text, and the phoneme error tendency vector are input, and the three sets are clustered in a predetermined number of classes using the similarity between the phoneme error tendency vectors as a scale. A clustering process for obtaining a centroid that is an average vector of the phoneme error tendency vectors in the cluster and outputting the cluster and the centroid;
Based on the cluster and centroid as input, based on speech and speech recognition result text included in each cluster, a base acoustic model adaptation process for generating an adapted acoustic model in which the base acoustic model is adapted for each cluster;
A post-adaptive acoustic model recording process for recording the post-adaptation acoustic model for each cluster;
An acoustic model adaptation method comprising: The acoustic feature extraction of the recognition target speech is performed for each frame, the output probability of the acoustic feature of the frame is obtained for all the states of all phonemes included in the base acoustic model, and the maximum value of the output probability is calculated as the output probability of the frame. Divide by the sum to make the posterior probability of the first phoneme, arrange the phoneme posterior probabilities obtained by the above speech units for the average value for each phoneme of the posterior probability of the first phoneme as the posterior probability vector A recognition-time phoneme error tendency vector generation process for generating a recognition-time phoneme error tendency vector by subtracting a speech group posterior probability vector of the entire speech group obtained in advance input from the outside;
A post-adaptive acoustic model in which the similarity between the recognition phoneme error tendency vector and the centroid of a set of a plurality of clusters and a set recorded in the post-adaptation acoustic model recording unit according to claim 1 or 2 is applied. The nearest centroid selection process to select and output as a model,
A speech recognition process in which the recognition target speech is subjected to speech recognition processing based on the applied acoustic model and a speech recognition result text is output;
A speech recognition method comprising:   A program for causing a computer to function as the acoustic model adaptation device according to claim 1 or the voice recognition device according to claim 3.

Images