JP2001255888A - Speech recognition device, speech recognition method and storage medium stored with program to execute the method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize highly efficient candidate search while switching grammars for speech recognition. SOLUTION: When speech is inputted (S2), the length of inputted speech is measured (S3). When a grammar which is to be used to recognize the inputted speech is specified from an application program side, cummulative input speech lengths that are stored to be made correspond to the grammar identification information and the number of speech inputs are obtained and updated. Then, an average speech length is updated from the updated cummulative input speech lengths and the number of speech inputs (S4). In parallel with the updating process, an appropriate searching method is selected employing the size of the specified grammar and the average speech length in the speech recognition processing conducted in the past using the grammar (S5). Then, a candidate search is conducted with the inputted speech as a recognition process by using the selected searching method (S6) and a recognition result is outputted (S7).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus having a plurality of grammars and a plurality of candidate search methods, a speech recognition method, and a storage medium storing a program for executing the method. .

[0002]

2. Description of the Related Art In speech recognition, for example, a grammar network based on a finite state automaton is used to extract word candidates that are considered to match input speech. Each phoneme is assigned to a state, and the possibility of transition from phoneme x to phoneme y is expressed as a transition from state x to state y, and the grammar of the vocabulary to be recognizable can be used as a constraint from the beginning to the end of the utterance. A state transition model (finite state automaton) that covers the phoneme sequence (phoneme sequence) is provided, and a phoneme sequence similar to the phoneme sequence of the input voice is extracted from the model. For example, a feature quantity representing each phoneme determined by an acoustic model is sequentially compared with a feature quantity representing each phoneme sequentially cut out from the input speech. If the similarity is high, a high score is given, and the finite state automaton indicates. A cumulative score is obtained for each possible phoneme string, and a phoneme string having a high cumulative score is extracted as a recognition result candidate. Note that typical search methods for extracting candidate phoneme strings from finite state automata covering possible transition states include a breadth-first search method, a depth-first search method, and an A * search method. . Hereinafter, each is outlined. First, the breadth-first search method is a search method that is broad and shallow. As shown in FIG. 3, the search is performed such that the search for a plurality of possible hypotheses can be completed at substantially the same time while the search depth, that is, the hypothesis time length is fixed. The numerical values in FIG. 3 indicate the search order (the same applies to FIG. 4). Therefore, in principle, real-time processing can be performed regardless of the input speech length (from the beginning to the end of the utterance), so that the grammar scale (for example, a short sentence having an arbitrary number of digits or a small number of vocabularies) (for example, This is effective when the total number of states of the finite state automaton) is small. However, since the processing time increases in proportion to the grammar scale, it is not suitable for large vocabulary speech recognition. Further, in the depth-first search method, as shown in FIG. 4, the search is performed in the depth direction (the direction in which the hypothesis time length becomes longer).
When the end is reached, the process of returning to the immediately preceding branch point and searching for another hypothesis toward the end is repeated. Therefore, this search method is an advantageous method when it is necessary to control the depth of a hypothesis, and is suitable for recognition of continuous digits having a fixed number of digits. However, there are drawbacks that real-time processing cannot be performed, and the processing time increases in proportion to the grammar scale. In the A * search method, a candidate phoneme string having a large sum of an intermediate score actually calculated up to that point and a predicted score that would be calculated when the expansion is advanced from that point onward is extracted. . This search method is suitable for large vocabulary speech recognition because the calculation time is hardly affected by the grammar scale.
However, when the input voice length is long, the calculation time may be significantly increased, and is not suitable for continuous numeral voice recognition or the like. Therefore, in a speech recognition method that performs recognition processing while dynamically changing the grammar according to the scene or purpose, use a search method that is appropriate for the scale and nature of the grammar used in each scene. Therefore, it is desirable to always realize efficient search.

[0003]

However, in the prior art, when searching for candidates while switching the grammar for speech recognition, a plurality of search methods may be used depending on the scale or nature of the grammar to be used. There is a problem that can not be. An object of the present invention is to solve such a problem of the conventional technology. That is, an object of the present invention is to provide a speech recognition method or the like that can realize an efficient candidate search when searching for a candidate while switching a grammar for speech recognition.

[0004]

In order to solve the above-mentioned problems, the invention according to claim 1 is used in a speech recognition method capable of searching for a candidate while switching a grammar for speech recognition. It is characterized in that a plurality of search methods are properly used according to the scale of the grammar. According to a second aspect of the present invention, in the speech recognition method according to the first aspect, a plurality of grammars or search methods to be selected are displayed, and a grammar or a search method to be used is selected from the plurality of grammars or search methods. According to a third aspect of the present invention, in the speech recognition method according to the first aspect, a search method is automatically selected according to the scale of the grammar. The invention described in claim 4 is
4. The voice recognition method according to claim 1, wherein the input voice length is stored, and when the search method is automatically selected in the next and subsequent times, the voice length is added to a selection criterion and searched. The method is characterized by selecting a law. According to a fifth aspect of the present invention, in the speech recognition method according to any one of the first to fourth aspects, at least one of acoustic model information, phoneme duration model information, and grammar network information is searched for by a plurality. It is characterized by being commonly used in law. According to a sixth aspect of the present invention, in the speech recognition method capable of searching for a candidate by a plurality of search methods, a search by each of the plurality of search methods is performed sequentially or in parallel, and each search method obtained by each search method is obtained. A final candidate is obtained according to the average value of the candidate scores. According to a seventh aspect of the present invention, a program for executing the voice recognition method according to any one of the first to sixth aspects is stored. According to an eighth aspect of the present invention, there is provided a speech recognition apparatus using the speech recognition method according to any one of the first to sixth aspects, wherein a plurality of likelihood table calculation means or a phoneme duration control penalty table calculation means are provided. , Which are commonly used in the search method.

[0005]

Next, embodiments of the present invention will be described in detail with reference to the drawings. The embodiment of the present invention always enables efficient search by appropriately switching a plurality of search methods according to the scale of a grammar to be used. As a criterion for an appropriate search method in the embodiment of the present invention, a value representing the scale of the grammar (for example, the total number of states of the finite state automaton) is mainly used, but other factors affecting search efficiency are used. The input voice length is also considered. However, when performing real-time processing with frame synchronization, the processing is started before the utterance ends, so it is necessary to determine the search method to be used at the stage where the input voice length has not been determined. Cannot be used as a criterion for the search method. Therefore, each time a voice is input, its length (utterance time) is stored, and the average input voice length for each grammar (that is, scene) is calculated. Provide a function that can be used as a reference. In this speech recognition device, by sharing information and processing means that can be used in common among a plurality of search methods, an increase in the amount of memory due to the implementation of the plurality of search methods is suppressed. Furthermore, in situations where real-time processing is not required, such as in batch processing, or in situations where the grammar scale is small and there is ample processing time, the scores of candidates obtained by executing multiple search methods in parallel and by each search method By calculating final candidates in accordance with the weighted average value of, robust speech recognition is enabled.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a speech recognition apparatus according to one embodiment of the present invention. As shown in the figure, the speech recognition apparatus of this embodiment includes a speech input unit 1 including a microphone, an amplification circuit, an AD converter, etc., temporarily stores data related to an acoustic model and various grammatical information, and performs search processing. Memory 2 for performing the calculation for the above, an average voice length calculation unit 3 for acquiring the input voice length from the beginning to the end of the input voice information and calculating the average voice length for each grammar, a grammar scale and an average voice A search method selection unit 4 for selecting an appropriate search method based on length information, a candidate search unit 5 for searching for a candidate by a specified search method, a weighted average of scores of each candidate obtained from a plurality of search methods , A score totaling unit 6 for determining a final candidate,
A recognition result output unit 7 for displaying a recognition result or outputting the result on paper, a search method specifying unit 8 having a keyboard, a mouse, and the like to specify a search method, and an acoustic model storage for storing data relating to an acoustic model. And a grammar information storage unit 10 for storing various grammar information. As shown in the figure, the candidate search unit 5 includes unique search units corresponding to various search methods such as search method A, search method B, and search method C, and a common processing unit 11 common to those search methods.
Note that the search method A, the search method B, and the search method C include, for example, a breadth-first search method (One-Pass DP method), a depth-first search method, an A * search method, and the like. The average voice length calculation unit 3, search method selection unit 4, candidate search unit 5, and score totalization unit 6 have, for example, a common memory and a CPU that operates according to a program stored therein.
The acoustic model storage unit 9 and the grammar information storage unit 10 are realized by, for example, a hard disk device.

FIG. 2 shows an operation flow of this embodiment.
The operation will be described below with reference to FIG. Prior to this operation, the acoustic model storage unit 9 stores data relating to an acoustic model for obtaining a score for each phoneme. The grammar information storage unit 10 stores various grammar information. In the grammar information storage unit 10, the average speech length of the input speech input so far is calculated for each grammar by the average speech length calculation unit 3, and is stored in association with the grammar. First, as initialization processing prior to recognition,
Data relating to the acoustic model is read from the acoustic model storage unit 9 into the work memory 2 and grammatical information (for example, a finite state automaton) is read from the grammatical information storage unit 10 into the work memory 2 (step S1). Thus, at this time, the scale of each grammar, for example, the total number of states of the finite state automaton, is determined. In such a state, in an actual voice recognition scene, voice is input from the microphone constituting the voice input unit 1 (step S2). Next, the average voice length calculation unit 3 measures the length of the input voice via the work memory 2 (step S3). Further, before or after that, when a grammar to be used for recognizing the input speech is specified from the application program side, the average speech length calculation unit 3 acquires grammar identification information indicating the grammar, and acquires the grammar identification information. The cumulative input voice length and the number of voice inputs corresponding to the voice recognition processing using the grammar stored in association with the identification information are acquired (step S4). Then, the input voice length measured at that time is added to the cumulative input voice length,
By increasing the number of voice inputs by one, the average voice length is updated from the updated cumulative input voice length and the updated number of voice inputs (step S4). In parallel with the process of updating the average voice length, the search method selection unit 4 uses the designated scale α of the grammar and the average voice length β in the voice recognition process performed in the past using the grammar. An appropriate search method is selected (step S5). As an example, when two types of search methods are implemented, an A * search method suitable for recognizing large vocabulary and short utterances, and an One-PassDP method suitable for recognizing long utterances with small vocabulary. Consider a rating scale. In this case, for example, an evaluation value S calculated by the following equation (1) can be used. S = log α−k · log β (1) If the evaluation value S is equal to or more than a certain value S0, the A * search method is used. If not, One-PassDP is used.
Use the law. Note that the values of S0 and k are constants determined experimentally. When the value of k is 0, a search method is selected using only the scale of the grammar as a criterion. As the value of k is increased, the degree of consideration of the average utterance length increases.

In the above embodiment, when the search method to be used is specified together with the grammar from the application program side, or when specified by the search method specifying unit 8 shown in FIG. 1, this evaluation is not performed. Follow the instructions. When the search method is specified by the search method specifying unit 8, a plurality of search methods are displayed, and one of the displayed search methods is selected by a mouse or the like. When driving the n kinds of search methods sequentially or in parallel, the n kinds of search methods are selected in order from the appropriate one, and the score totaling unit 6 weights the scores of the recognition results performed using them. The average is determined, and the candidate with the largest weighted average is used as the final search solution (recognition result). On the other hand, when searching with one selected search method,
Next, the candidate search unit 5 performs a candidate search as input voice recognition processing using the selected search method (Step S).
6). Using a grammar network based on a finite state automaton, phoneme string candidates that are considered to match the input speech are extracted. That is, data determined by the acoustic model and data extracted from individual phoneme data sequentially cut out from the input speech are sequentially compared using the work memory 2, and a higher score is given when the similarity is higher, and the finite state automaton indicates a higher score. A cumulative score is obtained for each phoneme string, and a phoneme string having the highest cumulative score is extracted as a recognition result candidate. In this case, a search method such as On
If the e-Pass DP method is selected, the search is performed so that the search depth, that is, the length of the phoneme string is constant, and the search for the target phoneme string that can be performed is completed almost simultaneously. And finally,
The search result, that is, the recognition result is output to the recognition result output unit 7 (step S7).

In the above description, the search method is selected based on the scale of the grammar and the input voice length. However, the search method may be selected based only on the scale of the grammar. When determining the grammar to be used, a plurality of grammars that can be used may be displayed, and one of the displayed grammars may be selected by a mouse or the like. In the above description, data relating to the acoustic model, that is, acoustic model information, and grammar network information based on the finite state automaton are commonly used in a plurality of search methods. Alternatively, one of the acoustic model information and the grammar network information may be commonly used in a plurality of search methods. It is also possible to use the phoneme duration model information in common for a plurality of search methods. In the A * search method performed asynchronously with the frame, the connection point between the area of the intermediate score (cumulative score) and the area of the prediction score, that is, the position (time t) of the connection representative point that is the evaluation value calculation point and the area of the intermediate score area The connection range is determined using the average time length and variance of phonemes,
The phoneme duration model information is information used for this determination. The common processing unit 11 commonly used in a plurality of search methods includes both or any of a likelihood table calculation unit and a phoneme duration control penalty table calculation unit. Note that the likelihood table calculation means means that each phoneme constituting a possible phoneme sequence represented by a finite state automaton using the likelihood table is a likelihood (probability) that is a phoneme constituting each input speech. , Similarity, and similarity), and a likelihood table is created using acoustic model parameters. Further, the phoneme duration control penalty table calculation means, using a phoneme duration control penalty table,
It is a calculating means for calculating the cumulative score f (t) at the elapsed time τ from the cumulative score f ′ (t) up to the connection representative point and the elapsed time τ from the connection representative point, and creating a penalty table thereof. The penalty table is obtained by giving a penalty αd (τ) shown in the following equation (2) in a table format in association with the elapsed time τ from the connection representative point. The product score f (t) is represented by the following equation (2). log f (t) = log f '(t) + αd (τ) (2)

[0010] In the A * search method, a two-stage search is performed in which a prediction score is obtained from the front to the rear of a phoneme sequence in a first stage, and then a candidate search is performed in a reverse direction in a second stage. In the One-PassDP method, literally, one search is performed from the front to the back of the phoneme string.
Focusing on the individual hypotheses at that time, after entering the state of the acoustic model corresponding to the phoneme x at a certain time, it stays in the phoneme for a certain time and then shifts to the next phoneme y. Therefore, by storing the time at which a phoneme preceding x shifts to x for each hypothesis, at the moment when a certain hypothesis shifts from phoneme x to phoneme y, for how long the hypothesis stays at phoneme x. It is possible to calculate information indicating whether or not the user has been. Then, the cumulative score of the hypothesis up to that point is added to the penalty calculated from the residence time at the phoneme x, and the cumulative score calculated is used for candidate selection, thereby improving the accuracy of speech recognition. In other words, such a penalty calculation method is called A
* Since it is the same as that of the search method, the penalty table calculation means for controlling the phoneme duration can be shared by a plurality of search methods. Although the speech recognition apparatus shown in FIG. 1 has been described above, a configuration without the score totalizing unit 6 and the search method designating unit 8 is also possible. Also,
A program for implementing the speech recognition method of the present invention as described above may be stored in a machine-readable storage medium. In this case, a machine-readable storage medium storing a program for performing the voice recognition method of the present invention, which is removable from the information processing apparatus, can be obtained. By attaching to an information processing apparatus that has not been able to perform the voice recognition by the voice recognition method of the present invention, the information processing apparatus can also perform the voice recognition by the voice recognition method of the present invention.

[0011]

As described above, according to the first aspect of the present invention, when searching for candidates while switching the grammar for speech recognition, a plurality of search methods are used according to the scale of the grammar to be used. Since it can be used properly, an efficient candidate search can be realized. According to the invention described in claim 2, in the invention described in claim 1, a plurality of grammars or search methods to be selected are shown, and a grammar or search method to be used is selected from the plurality of grammars or search methods. A grammar or a search method can be easily selected. According to the third aspect of the present invention, in the first aspect of the present invention, a search method is automatically selected according to the scale of the grammar, so that an appropriate search method can be more easily selected. it can. Also,
According to the invention described in claim 4, in the invention described in one of claims 1 and 3, the input speech length is stored, and when the search method is automatically selected in the next and subsequent times, Since the search method is selected by adding the voice length to the selection criterion, a more appropriate search method can be selected. According to the invention described in claim 5, according to claims 1 to 4,
In the invention described in (1), at least one of acoustic model information, phoneme duration model information, and grammar network information is commonly used in a plurality of search methods, so that a plurality of search methods can be easily realized and used. The amount of memory to be performed is small. According to the invention described in claim 6, when searching for a candidate, the search by each of the plurality of search methods is executed sequentially or in parallel, and according to the average value of the scores of each candidate obtained by each search method. Since final candidates are required, more accurate speech recognition can be realized depending on the situation. According to a seventh aspect of the present invention, there is provided a machine-readable storage medium storing a program for performing the voice recognition method according to any one of the first to sixth aspects. A storage medium detachable from the apparatus can be obtained, so that the storage medium is mounted on an information processing apparatus that has not been able to perform voice recognition by the voice recognition method according to any one of claims 1 to 6. Accordingly, the information processing apparatus can also perform speech recognition by the speech recognition method according to one of the first to sixth aspects.
According to the invention described in claim 8, in the invention described in any one of claims 1 to 6, the likelihood table calculation means or the penalty table calculation means for controlling phoneme duration is determined by a plurality of search methods. Since they are used in common, it is easier to implement a plurality of search methods.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a speech recognition device according to one embodiment of the present invention.

FIG. 2 is a flowchart for explaining a voice recognition method according to one embodiment of the present invention.

FIG. 3 is an explanatory diagram for explaining a speech recognition method according to a conventional technique and an embodiment of the present invention.

FIG. 4 is an explanatory diagram for explaining another speech recognition method according to the related art and the embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 1 voice input section, 2 work memory, 3 average voice length calculation section, 4 search method selection section, 5 candidate search section, 6 score totaling section, 7 recognition result output section, 8 search method designation section, 9
Acoustic model storage unit, 10 grammar information storage unit, 11 common processing unit.

Claims (8)

[Claims] 1. A speech recognition method capable of searching for a candidate while switching a grammar for speech recognition,
A speech recognition method characterized by using a plurality of search methods according to the scale of a grammar to be used. 2. The speech recognition method according to claim 1, wherein a plurality of grammars or search methods to be selected are displayed, and a grammar or search method to be used is selected from the plurality of grammars or search methods. 3. The speech recognition method according to claim 1, wherein a search method is automatically selected according to the scale of the grammar. 4. The voice recognition method according to claim 1, wherein the input voice length is stored, and said voice length is selected when a search method is automatically selected from the next time. A speech recognition method characterized by selecting a search method in addition to a criterion. 5. The speech recognition method according to claim 1, wherein at least one of acoustic model information, phoneme duration model information, and grammar network information is shared by a plurality of search methods. A speech recognition method characterized by using. 6. In a speech recognition method capable of searching for a candidate by a plurality of search methods, a search by each of the plurality of search methods is executed sequentially or in parallel, and a score of each candidate obtained by each search method is calculated. A speech recognition method characterized by finding final candidates according to an average value. 7. A storage medium storing a program for performing the voice recognition method according to claim 1. Description: 8. A speech recognition device using the speech recognition method according to claim 1, wherein the likelihood table calculation means or the phoneme duration control penalty table calculation means is implemented by a plurality of search methods. A speech recognition device characterized by having a configuration commonly used.