JP2003140685A - Continuous voice recognition device and its program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a continuous voice recognition device, which determines a recognition result in an early stage with good real-time response and has high recognition precision and a light arithmetic processing load, even when a detailed sound model and language model are used for voice recognition, and its program. SOLUTION: The continuous voice recognition device is equipped with means 21 and 31 of storing a simple acoustic model, a simple language model, a detailed acoustic model, and a detailed acoustic model, a 1st path processing means 20 of making a forward search for a continuous voice by using the simple acoustic model and simple language model and generating a word start- end list consisting of respective words as candidates and information on their start-end time, and a 2nd path processing means 30 of making a forward search for the continuous voice according to the information on the start-end time within a range of words included in the word start-end list by using the detailed acoustic model and detailed language model and generating a word array corresponding to the continuous voice.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a continuous speech recognition apparatus for recognizing continuously uttered speech and generating a word string indicated by the uttered continuous speech and its program.

[0002]

2. Description of the Related Art Conventionally, the following two methods have been known as continuous speech recognition methods for recognizing continuously uttered speech and generating a word string indicated by the speech. The first method is Imai et al., “Early determination of speech recognition result by successive comparison of the most likely word sequence”, IEICE Transactions, J84-D-II, Vol. 9, 1942-1949 (2000). ), The following processing is performed via two paths.

In the first pass, a detailed acoustic model and a simple language model are used to perform a search from the beginning of a sentence toward the end of the sentence (hereinafter referred to as a forward search) to obtain a plurality of word strings as recognition candidates. Next, in the second pass, the scores are updated for the plurality of word strings obtained in the first pass using the detailed language model, and the word string giving the maximum score is adopted as the recognition result.

The second method is Long Nguyen et al., "High-performance 2-pass N-best decoder", Minutes of DARPA Speech Recognition Workshop, pages 100-103, (199).
7) (Long Nguyen, et al., "Ef.
ficent 2-pass N-best deco
der ”, Proceedings of the DA
RPA Speech Recognition Wo
rkshop, pp. 100-103 (1997)).

First, in the first pass, a forward search is performed using a simple acoustic model and language model to create a list of words that are candidates for recognition and their end times. Next, under the restriction of the word end list, a search from the end of the sentence to the beginning of the sentence (hereinafter, referred to as backward search) is performed using the detailed acoustic model and language model in the second pass.

[0006]

However, in the first conventional method, since the detailed acoustic model is used in the first pass, the processing amount for limiting the recognition candidates as the acoustic model is made more detailed. Is increased and the time until the recognition result is determined is prolonged, and in the second pass, the score is updated only within the range of the word string obtained in the first pass, so that the ability of the detailed language model is sufficient. I can't pull it out,
There was a problem that high recognition accuracy could not be obtained.

Further, in the second conventional method, since the backward search is performed in the second pass, unlike the normal forward search, a detailed language model from the end of the sentence toward the beginning of the sentence is required. The word end found in the first pass is likely to continue to become a word end candidate in a certain section, which may increase the amount of processing in the second pass, or when the recognition result is successively confirmed without waiting for the end of utterance. Since the second pass is a backward search, it is not possible to apply an optimal early decision method that uses uniqueness from the beginning of the candidate word string, which reduces recognition accuracy and is not suitable for real-time processing. There was a problem such as.

The present invention has been made in order to solve such a problem, and an object thereof is to realize a real-time property of deciding a recognition result early even when a detailed acoustic model and language model are used for speech recognition. The object of the present invention is to provide a continuous speech recognition device with high recognition accuracy and a small calculation processing load, and its program.

[0009]

In view of the above points, the invention according to claim 1 recognizes continuous uttered speech,
In a continuous voice recognition device for generating a word string corresponding to the continuous voice, a simple first acoustic model, a simple first language model, and a second acoustic model more detailed than the first acoustic model. , And means for storing a second language model that is more detailed than the first language model, and positive for the continuous speech using the simple first acoustic model and the simple first language model. Performing a search, a first for generating a word start list consisting of information of each word that has reached a word end as a candidate for generating the word string and information of the start time at which each candidate word is uttered 1
Using the path processing means, the detailed second acoustic model, and the detailed second language model, each candidate word within the range of each candidate word included in the word start list A second pass processing means for performing a forward search for the continuous voice based on information on the start time at which the voice is uttered, and generating a word string corresponding to the continuous voice. There is.

With this configuration, the words to be searched by the second pass processing means and their start times are highly accurately limited by the word start list, and the continuous word ends are likely to have a common word start. The word start list has less redundancy than the word end list, and it is possible to improve word recognition accuracy without increasing the overall processing amount even when using a more detailed acoustic model or language model. A voice recognition device can be realized. In addition, the second pass processing means can apply an optimum early determination method that does not decrease the recognition accuracy in principle by utilizing the uniqueness of the candidate word string from the beginning of the sentence in order to perform a forward search from the beginning of the sentence toward the end of the sentence. Suitable for real-time processing.

Further, in the invention according to claim 2, in claim 1, the first pass processing means further includes information of a word which has reached near a word end during a forward search in the first pass processing means. , The information of the start time at which the word reaching the vicinity of the end of the word is uttered is additionally registered in the word start list. With this configuration, not only the word that has reached the end of the word but also the word that has reached the vicinity of the end of the word are subjected to the processing after the forward search in the second pass processing means, and continuous speech that enables more highly accurate speech recognition. A recognition device can be realized.

Further, in the invention according to claim 3, in claim 1, the first path processing means further additionally registers a word average score of each of the words included in the word beginning list in the word beginning list. Then, the second pass processing means further limits the candidate words to those in which the word average score of each of the words is a predetermined value or more, and the words corresponding to the continuous speech with respect to the limited words. It has a configuration for generating columns. With this configuration, since the words to be processed by the second pass processing means are limited, it is possible to realize a continuous speech recognition device that can reduce the processing load.

Further, in the invention according to claim 4, in claim 1, the second pass processing means further starts a predetermined time within a certain range before and after the start time at which each of the candidate words is uttered. As a time point, the second path processing means is configured to perform a forward search. With this configuration, a predetermined time within a certain range before and after the start time of each candidate word is added as the start time, and the forward search is performed by the second pass processing means, so that voice recognition can be performed with higher accuracy. A continuous voice recognition device can be realized.

According to a fifth aspect of the present invention, in the first aspect, the second pass processing means is configured to perform the first pass processing means even before the generation of the word start list is completed. As soon as the information of the candidate word and the information of the start time thereof are generated by the forward search by the one-pass processing means, the forward search is performed by the second-pass processing means, and the word string corresponding to the continuous speech is obtained. It has a configuration for performing processing for generation. With this configuration, the forward search processing is performed by the second pass processing means as soon as the information of the candidate word and the information of the start end time thereof are generated by the forward search by the second pass processing means. It is possible to implement a continuous speech recognition device suitable for real-time processing by applying an early determination method that does not reduce the noise.

Further, in the invention according to claim 6, in claim 1, the second pass processing means, after the generation of the word beginning list by the first pass processing means is completed,
The second path processing means is configured to start a forward search and perform processing for generating a word string corresponding to the continuous speech. With this configuration, even when real-time processing is not required, the load of arithmetic processing is small, and a continuous speech recognition device capable of improving word recognition accuracy can be realized.

According to a seventh aspect of the present invention, there is provided a program according to the first aspect, which causes a computer to recognize a uttered continuous voice and execute a process for generating a word string corresponding to the continuous voice. , To the computer,
Store a simple first acoustic model, a simple first language model, a second acoustic model more detailed than the first acoustic model, and a second language model more detailed than the first language model And the information of each word that is a candidate for generating the word string by performing a forward search on the continuous speech using the simple first acoustic model and the simple first language model. And a detailed second acoustic model and a detailed second language model, a first pass processing step for generating a word start list including the start time at which each candidate word is uttered. Using the, within the range of each of the candidate words included in the word start list, based on the information of the start time at which each candidate word is uttered, perform a forward search for the continuous speech. , The above It has a configuration to execute a second pass processing steps for generating a word string corresponding to the connection sound.

With this configuration, the word to be searched in the second pass processing step and its start time are highly accurately limited by the word start list, and the continuous word ends are likely to have a common word start. The word start list has less redundancy than the word end list, and it is possible to improve word recognition accuracy without increasing the overall processing amount even when using a more detailed acoustic model or language model. A voice recognition program can be realized. Also, the second
In the pass processing step, a forward search is performed from the beginning of the sentence toward the end of the sentence.Therefore, it is possible to apply an optimal early determination method that uses the uniqueness of the candidate word string from the beginning of the sentence and does not decrease the recognition accuracy in principle, and it can be used for real time processing Are suitable.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION A continuous speech recognition apparatus according to a first embodiment of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a block diagram showing a schematic configuration of a continuous speech recognition device 100 according to the first embodiment of the present invention. The continuous speech recognition device 100 includes an acoustic analysis unit 10 that acoustically analyzes an input voice to generate an acoustic analysis result, a first pass processing unit 20 that generates a word start list according to the acoustic analysis result, and an acoustic analysis result and a word. The second pass processing unit 30 generates a recognized word string using the start list.

The first pass processing unit 20 further includes a pronunciation dictionary / simple model storage unit 21, a tree-structured phoneme network generation unit (hereinafter referred to as a tree-structured phoneme NW generation unit) 22, an acoustic score calculation unit 23, and a language score. Calculation unit 24, and the first
It is configured by the forward search unit 25. The pronunciation dictionary / simple model storage unit 21 is used for the speech recognition processing in the first pass, a pronunciation dictionary, a simple acoustic model (hereinafter referred to as a simplified acoustic model), and a simple language model (hereinafter referred to as a simplified language model). ) Etc. is a component for storing. Here, it is needless to say that "simple" means that the scale of the model is small, and includes, for example, a model having a small number of states.

The tree-structured phoneme NW generation unit 22 receives as input the acoustic analysis result of the input voice output from the acoustic analysis unit 10, the pronunciation dictionary stored in the pronunciation dictionary / simplified model storage unit 21, the simplified acoustic model, and the like. , A phoneme network having a tree structure (hereinafter referred to as a tree structure phoneme network) is generated according to the acoustic analysis result of the input speech, and the generated tree structure phoneme network is an acoustic score calculation unit 23, a language score calculation unit 24, And the first forward search unit 25.

The acoustic score calculation unit 23 receives the acoustic analysis result of the input voice, the simple acoustic model and the tree-structured phoneme network as inputs, and calculates the acoustic score for the acoustic analysis result using the simple acoustic model and the tree-structured phoneme network, It is a component for outputting to the first forward search unit 25. Here, as a simple acoustic model, for example, a triphone hidden Markov model (Hidden) with a small number of states is used.
Markov Model, hereinafter referred to as HMM. )
Etc. can be used. The method of calculating the acoustic score is publicly known, and the description thereof will be omitted.

The language score calculation unit 24 receives the simple language model and the tree-structured phoneme network as input, calculates the language score for the active node on the tree-structured phoneme network using the simplified language model, and outputs the first forward search unit. 25 is a component for outputting to 25. Here, for example, a word bigram or the like can be used as the simple language model. The method of calculating the language score is well known, and the description thereof will be omitted.

The first forward search unit 25 receives the tree-structured phoneme network, the acoustic score, and the language score,
Propagate active nodes forward using acoustic and language scores on a tree-structured phoneme network,
This is a component for creating a word start list that is a list of words that have not been pruned and remains until the end of the word and their start times, and output the created word start list to the second pass processing unit 30. Note that "forward" refers to the direction from the beginning of the sentence to the end of the sentence, and hereinafter, the forward search is referred to as "forward search".

Here, the tree-structured phoneme network may be a static one that is used by looping one tree-structured phoneme network or a dynamic one that is obtained by connecting a plurality of tree-structured phoneme networks. . In addition, in order to make the word start list highly accurate, the first forward search unit 25 performs the word pair approximate search depending on the immediately preceding word.

On the other hand, the second pass processing unit 30 further includes a pronunciation dictionary / detailed model storage unit 31 and a linear structured phoneme network generation unit (hereinafter referred to as a linear structured phoneme NW generation unit).
32, acoustic score calculator 33, language score calculator 34,
And the second forward searching unit 35. The pronunciation dictionary / detailed model storage unit 31 is used as a pronunciation dictionary, a detailed acoustic model (hereinafter referred to as a detailed acoustic model), and a detailed language model (hereinafter referred to as a detailed language model) used in the speech recognition processing in the second pass. ) Etc. is a component for storing. Here, of course, "detailed"
It means that the scale of the model is larger than a certain level, and includes, for example, a model having a large number of states.

The linear structured phoneme NW generation unit 32 receives the acoustic analysis result of the input voice output from the acoustic analysis unit 10, the pronunciation dictionary stored in the pronunciation dictionary / detailed model storage unit 31, the detailed acoustic model, and the like as inputs. , A phoneme network having a linear structure (hereinafter referred to as a linear structure phoneme network) is generated according to the acoustic analysis result of the input speech, and the generated linear structure phoneme network is used as the acoustic score calculation unit 3
3, language score calculation unit 34, and second forward search unit 3
5 is a component for outputting each to 5.

The acoustic score calculation unit 33 receives the acoustic analysis result of the input voice, the detailed acoustic model and the linear structured phoneme network as an input, and outputs an acoustic score for the acoustic analysis result.
It is a component for calculating using the detailed acoustic model and the linear structured phoneme network and outputting it to the second forward searching unit 35. Here, for example, a triphone HMM having many states can be used as the detailed acoustic model. The method of calculating the acoustic score is publicly known similarly to the method of calculation by the acoustic score calculator 23, and the description thereof will be omitted.

The language score calculation unit 34 receives the detailed language model and the linear structured phoneme network as inputs, and calculates the language score for the active word start node on the linear structured phoneme network using the detailed language model,
It is a component for outputting to the forward search unit 35. Here, for example, a word trigram can be used as the detailed language model. The method of calculating the language score is publicly known similarly to the method of calculation by the language score calculator 34, and the description thereof will be omitted.

The second forward searching section 35 has a linear structured phoneme network, an acoustic score output from the acoustic score calculating section 33 (hereinafter referred to as a second acoustic score), and a language score output from the language score calculating section 34. (Hereinafter, referred to as a second language score), and the first forward search unit 25.
It is a component for determining a recognized word string by inputting the word start list output from, and outputting it to the outside of the continuous speech recognition apparatus 100.

At this time, the second forward searching unit 35 forwards the active node on the linear structured phoneme network to the forward direction only by limiting the words included in the word start point list and the start time, and the above second acoustic score and It is assumed that the second language score can be used to perform early confirmation using uniqueness from the beginning of a word string candidate without waiting for the end of utterance. Of course, after the utterance ends, the second forward searching unit 35
It is also possible to perform the processing in.

When the word trigram is used for the detailed language model, the second forward search section 35 can perform the 1-best search which advances the search while saving one optimum word history for each immediately preceding word. To do. Also, the reason why the linear structured phoneme network is used is that it is not necessary to share one phoneme node among a plurality of words because the words to be activated differ according to the word start list at each time.

As the continuous speech recognition device 100, in addition to the above-described one, the following embodiments can be used. (1) The first forward searching unit 25 additionally registers not only the word end but also the word reaching the end of the word and its start time in the word start list, and the second forward searching unit 35 regards the additionally registered words. An embodiment in which the above processing is also performed. (2) The first forward searching unit 25 additionally registers the word average score in the word start list, and the second forward searching unit 35 limits the additionally registered word average score to a value exceeding a predetermined threshold. An embodiment of pruning and performing the above processing. (3) The second forward searching unit 35 adds a predetermined time within a time range of a certain width to the start time registered in the word start end list as a start time, and widens the search range to allow the word search to start. Embodiment.

Regarding the execution order of the processing in the first pass processing unit 20 and the second pass processing unit 30, the following two embodiments are possible. (1) Implementation for real-time processing in which the second pass processing unit 6 is processed in parallel with a certain delay time during the processing of the first pass processing unit 20 and the word is confirmed early without waiting for the end of the utterance. Form. This is because, even before the word start list is generated and completed by the first pass processing unit 20, as soon as a predetermined candidate word and the start time information thereof are generated,
The second pass processing unit 30 performs a forward search to perform processing for generating a word string corresponding to continuous speech.

(2) An embodiment in which the processing in the second path processing unit 30 is started after the processing in the first path processing unit 20, that is, after the utterance, when the real-time processing is not required. This is because the second pass processing unit 20 starts a forward search in the second pass processing unit after the word start list is generated by the first pass processing unit, and generates a word string corresponding to continuous speech. Is to be processed.

The processing in the continuous speech recognition apparatus 100 according to the first embodiment of the present invention will be described below with reference to the drawings. 2 and 3 are flowcharts showing the flow of processing in the first pass processing unit 20 of the continuous speech recognition device 100 according to the first embodiment of the present invention. The continuous voice recognition device according to the first embodiment can be configured by a computer device having a general configuration including an interface, a control / calculation device, and a storage device, which are not shown. In that case, the pronunciation dictionary / simple model storage unit 21 and the pronunciation dictionary / detailed model storage unit 31
Corresponds to the storage device, and the other first pass processing units 20
The components of the second pass processing unit 30 and the acoustic analysis unit 10 can be associated with a control / calculation device.

In step S210, the first pass processing unit 20
Performs initialization processing in which the processing time t of the target input speech is set to 0, only the phoneme node corresponding to the initial word <s> is activated, and its total score is set to 0.
In step S220, the first pass processing unit 20 retrieves the acoustic analysis result 32 of the input voice at the time t from the acoustic analysis unit 10.

In step S230, the first pass processor 20
Selects one node from all active nodes and sets it as node n. In step S240, the acoustic score calculation unit 23 calculates a simple acoustic score for the input speech at the node n and time t, and the first forward searching unit 25 calculates the total score of the node n selected at step S230 for the input speech at time t. The simple acoustic score of is added.

In step S250, the language score calculation unit 2
4 calculates the simple language score of the node n, and the first forward search unit 25 updates the simple language score of the total score of the node n each time the node transits. The simple language score of the total score of the node n is updated each time the node transits because the tree-structured phoneme network with high search efficiency is used for the phoneme network, and one node is shared by multiple words. It is a thing.

In step S261, the first forward search unit 2
5 determines whether the total score of the node n is less than or equal to the pruning threshold value. When it is determined in step S261 that the total score of the node n is less than or equal to the pruning threshold, the first forward search unit 25 deactivates the node n in step S262 and proceeds to step S270.

When it is determined in step S261 that the total score of the node n exceeds the pruning threshold value, the process proceeds to step S263. In step S263, the first forward search unit 25 determines whether the node n is the end of the word w. When it is determined in step S263 that the node n is the end of the word w, the process proceeds to step S264. Even if the node n is not the end of the word w, if the node n has a sufficiently high score near the end of the word, the processing is performed in step S26.
There may be embodiments that proceed to No. 4, but they are not excluded.

In step S264, the first forward search unit 2
5 adds the word w to which the node n belongs, its start time, and the word average score to the word start list, and if the same word has already been registered at the same start time, updates it to the larger word average score. In step S265, the first forward search unit 25 activates all leading nodes of subsequent words.

If it is determined in step S263 that it is not the end, the process proceeds to step S266 and step S266.
At 266, the first forward searching unit 25 activates all subsequent phoneme nodes. Step S262 above,
When the process in either step S265 or step S266 is completed, the process proceeds to step S2.
Proceed to 70.

In step S270, the first forward search unit 2
5 determines whether or not the processing of all active nodes is completed, and when it is determined that the processing is completed, the processing is step S.
If it is determined that the processing has not ended, the processing returns to step S230, the next active node is selected, and the above processing is repeated.

In step S280, the first forward search unit 2
5 determines whether or not the processing for all input voices is completed. If it is determined to be completed, the processing for the first pass is completed, and if it is determined that it is not completed, the processing is step S290. Move on to. In step S290, first forward search unit 25 adds 1 to time t, and then the process returns to step S220, and the above process is repeated for the input voice at time t + 1.

FIGS. 4 and 5 show the second pass processing section 3 of the continuous speech recognition apparatus 100 according to the first embodiment of the present invention.
7 is a flowchart showing a flow of processing in 0. Hereinafter, the processing in the second pass processing unit 30 of the continuous speech recognition device 100 according to the first embodiment of the present invention will be described with reference to the drawings.

In step S410, the second pass processing unit 30
Performs initialization processing in which the processing time t of the target input speech is set to 0, only the phoneme node corresponding to the initial word <s> is activated, and its total score is set to 0.
In step S420, the second pass processing unit 30 extracts the acoustic analysis result 32 of the input voice at the time t from the acoustic analysis unit 10.

In step S430, the second pass processing unit 30
Selects one node from all active nodes and sets it as node n. In step S440, the acoustic score calculation unit 33 calculates a detailed acoustic score for the input speech at the node n and time t, and the second forward search unit 35 calculates the total score of the node n selected at step S430 for the input speech at time t. Add the detailed acoustic score of.

In step S451, the second forward search unit 3
5 determines whether the total score of the node n is less than or equal to the pruning threshold value. When it is determined in step S451 that the total score of the node n is less than or equal to the pruning threshold value, the second forward search unit 35 deactivates the node n in step S452 and proceeds to step S470.

If it is determined in step S451 that the total score of the node n exceeds the pruning threshold value, the process proceeds to step S453. Note that in step S451,
There may be an embodiment in which the word average score registered in the word start list obtained by the first pass processing unit 20 is also used in the pruning determination of the head node of the word, and it is not excluded.

In step S453, the second forward search unit 3
5 determines whether the node n is the end of the word w. When it is determined in step S453 that the node n is the end of the word w, the process proceeds to step S454, and when it is determined that it is not the end, the process proceeds to step S456.

In step S454, the second forward search unit 3
5 refers to the word start list and activates the head nodes of all words that can start at time t + 1. Of course, in addition to the word start time registered in the word start list, an embodiment is also possible in which the word search start is allowed with a certain width before and after. In step S455, the second forward search unit 3
The step 5 adds the detailed language score to the total score of the nodes activated in step S454.

In step S456, the second forward searching unit 3
5 activates all subsequent phoneme nodes. Upon completion of the processing in any of the above step S452, step S455, or step S456, the processing proceeds to step S460.

In step S460, the second forward searching unit 3
5 determines whether or not the processing of all active nodes is completed, and when it is determined that the processing is completed, the processing is step S.
If it is determined that the processing has not ended, the processing returns to step S430, the next active node is selected, and the above processing is repeated. In step S470,
The second forward searching unit 35 refers to the word histories of all the active nodes, and if there is a section in which the word string following the initial word <s> is unique, determines it early as a part of the recognition result.

In step S480, the second forward search unit 3
5 determines whether or not the processing for all the input voices is completed. If it is determined that the processing is completed, the processing for the second pass is completed. If it is determined that the processing is not completed, the processing is step S490. Move on to. In step S490, second forward search unit 35 adds 1 to time t, and then the process returns to step S420, and the above process is repeated for the input voice at time t + 1.

If the voice recognition is not required to have real-time characteristics, the word history is traced from the end-of-sentence word </ s> after the processing in step S480 is completed without early determining the word in step S470. An embodiment is also possible in which the back is performed and the word string of the entire utterance is output at one time.

The operation of the first forward searching section 25 of the present invention will be described with reference to FIG. The beginning word <s> has time 0 as a start end and times 2, 3, and 4 as end candidates. The word w1 follows the initial word <s> when the end of the initial word <s> is set to time 3, and has time 4 as a starting end and times 8 and 9 as end candidates. The word w2 follows the beginning word <s> when the end of the beginning word <s> is time 2, and has the beginning time 3 but is pruned at time 8.

The word w3 follows the sentence head word <s> when the sentence start word <s> ends at time 4, and has time 5 as a start end and times 12 and 13 as end candidates. Furthermore, the word w3 can follow the word w1 when the time 8 is the end, and the time w and the time w are the times 12 and 13 with the time 9 as the start.
Is the candidate for the termination. The word w4 is subsequent to the word w1 when the time 8 is the end, and the time 9 is the start and the times 13, 14, and 15 are the candidates for the end.

FIG. 7 shows the first forward searching section 25 shown in FIG.
It is a figure which shows an example of the word starting point list created by. For word candidates starting at time 0, the first word <s>
And the word average score is listed in parentheses.
, The word average score is −59. Similarly, the word candidate starting at time 4 has word w1, the word candidate starting at time 5 has word w3, and the word candidate starting at time 9 has words w3 and w4. Since the word w2 was pruned before reaching the end of the word, it is not included in the start-of-word list.

FIG. 8 is a diagram for explaining the operation of the second forward searching section 35 which operates under the constraint of the word start list shown in FIG. The initial word <s> becomes active at time 0 and a forward search is started. Note that the word w1 becomes active at time 4 and the forward search is started,
There may be an embodiment in which the forward search can be started from time 3 and time 5 before and after time 4.

Similarly, the word w3 becomes active at time 5 and time 9 and times before and after them, and the word w4 becomes active at time 9 and times before and after that, and the forward search is started. As described above, since the word to be searched by the second pass processing unit 30 and the search start time are highly accurately limited by the word start list, even if a more detailed acoustic model or language model is used, the total processing amount is reduced. It is possible to improve the accuracy of the correct answer of the word without increasing.

As described above, the continuous speech recognition apparatus and the program therefor according to the first embodiment of the present invention utilize the list of candidate words obtained by a simple model and their start time points in a detailed forward-looking manner. Since the search is performed, the accuracy of a word can be improved without increasing the overall processing amount even when a more detailed acoustic model and language model are used. In addition, in the second pass as well, in order to perform a forward search from the beginning of the sentence toward the end of the sentence, it is possible to apply an optimal early determination method using uniqueness from the beginning of the candidate word string, which is suitable for real-time processing. There is.

In the first embodiment of the present invention, the continuous speech recognition method for performing the processing in each of the steps S210 to S490 using the continuous speech recognition apparatus according to the first embodiment. However, it is also possible to use a predetermined computer in which a continuous voice recognition program for executing the continuous voice recognition operation including these steps S210 to S490 is installed.

In addition to the method of loading the above continuous speech recognition program stored in a predetermined storage medium into a computer, the present invention obtains the above continuous speech recognition program in a file format from a communication interface and a network, The same effect can be obtained by the method implemented by the computer. Furthermore, using a network makes it easy to update and distribute the program.

[0064]

As described above, according to the present invention, even when a detailed acoustic model and language model are used for speech recognition, the recognition result is excellent in real time and the recognition result is excellent.
It is possible to realize a continuous speech recognition device with high recognition accuracy and a low calculation processing load, and its program.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a continuous speech recognition apparatus according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a flow of processing performed in a first pass processing unit of the continuous speech recognition apparatus according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing in detail the flow of a part of the processing performed in the first pass processing unit of the continuous speech recognition device according to the first embodiment of the present invention.

FIG. 4 is a flowchart showing a flow of processing performed in a second pass processing unit of the continuous speech recognition device according to the first embodiment of the present invention.

FIG. 5 is a flowchart showing in detail the flow of a part of the processing performed in the second pass processing unit of the continuous speech recognition device according to the first embodiment of the present invention.

FIG. 6 is a diagram for explaining the operation of the first forward search section of the continuous speech recognition device according to the first embodiment of the present invention.

FIG. 7 is a diagram showing an example of a word start list created by a first forward search unit of the continuous speech recognition apparatus according to the first embodiment of the present invention.

FIG. 8 is a diagram for explaining the operation of the second forward searching section of the continuous speech recognition device according to the first embodiment of the present invention, which operates under the constraint of the word start list.

[Explanation of symbols]

10 Acoustic Analysis Department 20 1st pass processing unit 21 Pronunciation dictionary / simplified model memory 22 Tree structure phoneme NW generation unit 23 Acoustic Score Calculation Unit 24 Language score calculator 25 First Forward Search Unit 30 Second pass processing unit 31 pronunciation dictionary / detail model storage 32 Linear structured phoneme NW generator 33 Acoustic Score Calculation Unit 34 Language score calculator 35 Second Forward Search Unit 100 continuous speech recognizer

Claims (7)

[Claims] 1. A continuous speech recognition apparatus for recognizing spoken continuous speech and generating a word string corresponding to the continuous speech, comprising a simple first acoustic model, a simple first language model, Means for storing a second acoustic model that is more detailed than the first acoustic model and a second language model that is more detailed than the first language model, the simple first acoustic model, and the simple A forward search is performed on the continuous speech using the first language model, information of each word that has reached the end of a word as a candidate for generating the word string, and a starting point at which each candidate word is uttered. The first path processing means for generating a word start list including time information, the detailed second acoustic model, and the detailed second language model are used to include the word start list included in the word start list. Candidate Within each word, based on the information of the start time at which each of the candidate words is uttered, a forward search is performed on the continuous speech to generate a word string corresponding to the continuous speech. A continuous speech recognition apparatus comprising: two-pass processing means. 2. The first pass processing means further utters information of a word reaching the word end vicinity during the forward search in the first pass processing means and a word reaching the word end vicinity. The continuous voice recognition device according to claim 1, wherein information on a start time is additionally registered in the word start list. 3. The first pass processing means further additionally registers the word average score of each of the words included in the word start list to the word start list, and the second pass processing means further comprises: 2. The candidate words are limited to those having an average word score of each word of a predetermined value or more, and a word string corresponding to the continuous speech is generated for the limited words. Continuous speech recognizer. 4. The second pass processing means further employs a forward search in the second pass processing means, with a start time being a predetermined time within a certain range before and after the start time at which each of the candidate words is uttered. The continuous speech recognition apparatus according to claim 1, wherein 5. The second pass processing means becomes the candidate by the forward search in the first pass processing means even before the generation of the word beginning list by the first pass processing means is completed. When the word information and the start time information thereof are generated, a forward search is performed by the second pass processing means, and a process for generating a word string corresponding to the continuous speech is performed. Item 1. A continuous speech recognition device according to item 1. 6. The second pass processing means starts a forward search in the second pass processing means after the first pass processing means completes the generation of the word start list, and corresponds to the continuous speech. The continuous speech recognition apparatus according to claim 1, wherein the continuous speech recognition apparatus performs a process for generating a word string to be reproduced. 7. A program for causing a computer to execute a process for recognizing a uttered continuous voice and generating a word string corresponding to the continuous voice, the computer having a simple first acoustic model, a simple A first language model, a second acoustic model that is more detailed than the first acoustic model, and a second language model that is more detailed than the first language model, and the simple first Of the candidate words for generating the word string by performing a forward search on the continuous speech by using the acoustic model and the simple first language model and utterance of the candidate words. The first pass processing step for generating a word start list including the information of the generated start time, and the detailed second acoustic model and the detailed second language model Within the range of each of the candidate words included in the start point list, based on the information of the start point time when each of the candidate words is uttered, a forward search is performed for the continuous voice, and the continuous voice is supported. And a second pass processing step for generating a word string to be executed.