JP2817406B2 - Continuous speech recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a continuous speech recognition system for recognizing continuously uttered speech in an automatic interpretation system, a voice question answering (QA) system, and the like.

[0002]

2. Description of the Related Art A man-machine interface using voice is easy to use and natural because training such as a keyboard is not required. In addition, it is desired to handle not only speech uttered in units of words but also continuous speech such as conversational sentences because it is easier to use and more natural.

When such an interface using voice is used in an automatic interpreting system, a voice QA system, or the like, the main purpose is not to clarify each word of the input voice, but to extract the meaning of the input voice. That is, it is necessary to understand the meaning of the input voice.

[0004] Because the question answering system needs to know the meaning of the question in order to properly answer the user's question, the automatic interpreting system uses the language used for voice input in another language such as English. This is because it is necessary to clarify the meaning of the input sentence in order to convert the input sentence appropriately.

As a speech recognition method for extracting the meaning of an input speech as a natural language without re-analyzing the syntax and semantic analysis of the recognition result of the input speech, Japanese Patent Application No. Hei.
No. 72889, “Continuous speech recognition method” (hereinafter referred to as Document 1)
There is). In Document 1, the semantic relation between words in a word string received by an automaton for speech recognition and words is described.
When the word sequence is accepted by the automaton, the syntactic analysis and semantic analysis are performed again on the speech recognition result of the input speech by storing in advance the semantic relationship between the corresponding state transition sequences for each word. We propose a speech recognition method that extracts the meaning of input speech as a natural language.

[0006]

However, in the speech recognition method of Document 1, for example, the representation of the tense of the recognized sentence and the type of sentence such as a polite sentence or a question sentence in an intermediate expression is considered. Did not.

As a speech recognition method capable of handling a context-free grammar with the same processing amount as a method using a finite automaton, the 1989 Spring Meeting of the Institute of Electronics, Information and Communication Engineers, "Continuous Speech Recognition Using Extended Transition Networks" On the other hand, there is a formula (hereinafter referred to as Reference 2) (Kazuya Yoshida, Takao Watanabe). Speech recognition using an extended transition network (hereinafter referred to as ATN) is as follows: (1) Context-free grammar can be handled by calling a sub-network. (2) A process considering the history is performed by a register and a mechanism for testing the register, and word order freedom, co-occurrence relation, and dependency can be handled. Have a high ability to describe natural languages such as

[0008] However, in the speech recognition method of Reference 1, the case where the recognition network has a subnet as in Reference 2 has not been considered.

Also, in a spoken dialogue in Japanese, the subject is often omitted, such as "(you) go to school?" Or "(I) want a ticket for a concert in Basia." Although the omitted elements must be supplemented in the creation, the speech recognition method of Reference 1 does not consider such elements.

An object of the present invention is to provide a continuous speech recognition system which can overcome such drawbacks and recognize more natural and various sentences in speech recognition and can output a semantic expression thereof. It is in.

[0011]

According to the continuous speech recognition method of the present invention, a first storage means for storing a network relating to words expressing a grammar to be subjected to continuous speech recognition is combined with a standard pattern of words according to the network. Continuous speech recognition means for recognizing continuous speech, second storage means for storing a semantic relationship between a certain word and another word in the network and feature information for each word, and the continuous speech recognition means And an output unit for outputting a recognition result generated from the above and an intermediate expression from the second storage unit.

The continuous speech recognition means recognizes continuous speech by combining a predetermined standard pattern of a recognition unit in accordance with the network and a sub-network called from each network. A network expressing a grammar to be recognized and a sub-network called from each network are stored, and the output unit outputs a recognition result generated from the continuous speech recognition unit and a meaning between a certain word and another word in the second storage unit. And outputting an intermediate expression of the words based on the basic relation and the feature information for each word in the network and the sub-network.

[0013] The present invention further comprises: a second storage unit for storing feature information to be added to another word when a certain word is recognized; and an adding unit for adding the information to the intermediate expression. I do.

Further, when a certain word is recognized, feature information to be added to the word is extracted from the word dictionary storage unit, and the adding means adds the information to the intermediate expression.

The word dictionary storage unit stores words that do not exist in the network expressing the grammar to be subjected to continuous speech recognition but are always necessary to create an intermediate expression, and the output unit stores the words based on the information. In which an intermediate expression is created and output.

Further, the word dictionary storage unit stores a word that is supplemented when a specific word in a network expressing a grammar to be subjected to continuous speech recognition is recognized, and the output unit stores an intermediate word based on the information. It is characterized by creating and outputting an expression.

Further, the word dictionary storage unit stores a specific grammar of the continuous speech recognition target grammar to be deleted when a specific word in the network representing the grammar of the continuous voice recognition target is recognized. A word is stored, and the output unit creates and outputs an intermediate expression based on the information.

[0018]

Next, the operation of the present invention will be described.

FIG. 3 is a diagram showing an example of a semantic relation storage unit in one embodiment of the present invention, FIG. 4 is a diagram showing a first example of a voice recognition network, and FIG. 5 is a meaning for a main network in FIG. FIG. 6 shows an example of the contents of the relation storage unit.
FIG. 7 shows an example of the contents of the semantic relationship storage unit for the sub-network in FIG. 4, FIG. 7 shows a second example of the speech recognition network, and FIG. 8 shows an example of an intermediate representation of the network in FIG. FIG. 9 is a diagram showing an example of the contents of the semantic relationship storage unit for the network in FIG. 7, FIG. 10 is a diagram showing a third example of the network for speech recognition, and FIG. 11 is a semantic relationship for the network in FIG. FIG. 12 is a diagram showing an example of the contents of the storage unit, FIG. 12 is a diagram showing a fourth example of the voice recognition network, FIG. 13 is a diagram showing an example of an intermediate representation of the network in FIG. 12, and FIG. FIG. 16 is a diagram showing a fifth example, FIG. 16 is a diagram showing a sixth example of a voice recognition network, and FIG. 17 is a diagram showing an example of an intermediate representation of the network in FIG.

For example, it is assumed that a voice "I want a ticket for a concert in Bercia over the years" is recognized in the voice recognition network shown in FIG. Here, in FIG. 16, the numbers assigned to each node such as “I am” and “ticket” are called ID numbers. The intermediate expression of this sentence "I want a ticket for a Berthia concert in the future" is as shown in FIG. FIG.
7, the notation in the intermediate expression corresponding to the word in each speech recognition network such as "want"
Call. A notation expressing the relationship between words such as “benefit” and “object” is called CASE. In FIG. 17, information given beside each CP, such as #style and #noun semantic features, is referred to as feature information. #Sentence style is sentence pattern, #tense is time, #verb semantic classification is verb semantic classification, #noun semantic feature is noun semantic feature, #quantity is singular / plural, #goodness is sentence polite sentence It represents that. In addition, the characters and numerals in each of the triangles indicate the value of each feature information. In the present invention, for example, in FIG. 3, the semantic relationship between a certain word and another word in the network and the feature information for each word in the network are stored, and the intermediate expression as shown in FIG. Is to be created. As a result, the type of sentence such as the tense, the polite sentence, and the question sentence of the recognized sentence can be represented in the intermediate expression.

Consider a case where a speech recognition network such as ATN is used as in Reference 2. now,
Assume that recognition is performed using a main network and a sub-network as shown in FIG. In the main network of FIG. 4 (a), for example, at “sub date and time”, FIG.
Jumps to the sub-network indicated by. At this time, according to the invention of claim 2, for example, the table shown in FIG. 5 for the network of FIG.
6 (a) and 6 (d) for the network of FIG.
The tables shown in (b) and (c) are prepared. The subnet table is referred to by designating the subnet in the column of CP in the table of FIG. As a result, for example, A
An intermediate representation as shown in FIG. 17 can also be created from a speech recognition result using a network such as a TN having a main network and a subnetwork called therefrom.

In the voice recognition network shown in FIG. 7, for example, when the input voice is "I want one ticket", the intermediate expression is as shown in FIG. The value of #quantity in the feature information of the “ticket” is a SIN representing a single number. However, if the input voice is “I want two tickets”, this intermediate expression is as shown in FIG. 8B, and since there are two tickets, the value of the # quantity in the feature information of the CP “ticket” Is a PL representing a plurality.

At this time, in the invention of claim 3, since the CP information such as "one sheet" or "two sheets" determines the feature information of the parent CP "ticket", the "parent assignment information" of FIG. "
, Information to be added to the feature information of the parent CP is stored. As a result, it is possible to create an intermediate representation of a network as shown in FIG. 7 even when feature information is added to another word when a certain word is recognized.

By the way, for example, in the description of the feature information column in FIG. 3, the value of #noun semantic feature or #verb meaning classification is "I".
And information unique to words such as "tickets". Therefore, in the invention of claim 4, each feature information is described in a dictionary, and information such as #noun meaning semantic features and #verb meaning classification is supplemented by referring to the dictionary. As a result, the description of the table is simplified, and the storage capacity of the table can be reduced.

In the case of Japanese, for example, the subject of a sentence is often omitted. It is assumed that a voice, "I want a ticket for a concert in Bercia to come", which is one of the recognition results of the network in FIG. 10, is input. This intermediate representation must be the same as in FIG.
There is no word corresponding to “I” in the recognition result. At this time, in the invention of claim 5, although omitted in the voice, the necessary CP must be supplemented in the intermediate expression, so that, for example, the one in which the ID number column is D as shown in FIG. 11 is supplemented by default. Shall be. As a result, words that do not exist in the speech recognition result but are always necessary for creating the intermediate expression are supplemented,
An intermediate representation as shown in FIG. 17 can be created.

Assume that there is a voice recognition network as shown in FIG. Now, consider a case where the input sentence is, for example, "I want a Bercia ticket". In this case, semantically, "I want a ticket for the Bersia cosert." The word "concert" has been omitted. At this time, the intermediate representation is as shown in FIG.

It is also assumed that the input sentence is, for example, "I want a ticket for Yume no Yuminsha." In this case, semantically, "I want a ticket for Yume no Yuminsha theater", and the word "drama" is omitted. At this time, the intermediate representation is as shown in FIG. At this time, according to the invention of claim 6, when a certain word is recognized, a different meaning is supplemented for each word. As a result,
When a certain word is recognized, an appropriate word can be supplemented to create a correct intermediate expression.

Consider a case where there is a voice recognition network as shown in FIG. This network is, for example, "I want a ticket for a concert in Basia"
The input sentence "I want a ticket for a concert in Barcia" is also omitted. Each of the intermediate expressions is as shown in FIG.

At this time, if the "I" CP is supplemented by default using the invention of claim 5, there is no problem if "I am" is omitted in the speech recognition result, but "I am" in the speech recognition result. When "" exists, an erroneous intermediate expression having two trees indicating that the relationship between "want" and "I" is beneficiary is created.

Therefore, in the invention of claim 7, when a specific word is recognized, a part of the intermediate expression is deleted to create a semantic expression. As a result, a correct intermediate expression can be created in both cases where a word appears in the speech recognition result and where it is omitted.

[0031]

Next, an embodiment of a continuous speech recognition system according to the present invention will be described with reference to the drawings.

First, an embodiment of the present invention will be described. FIG. 1 is a configuration diagram showing one embodiment of the present invention. In FIG. 1, a voice input unit 101 converts an input voice into a digital signal, performs analysis, obtains a time series of feature vectors, and outputs the result to a continuous voice recognition unit 104. The network storage unit 102 stores a sentence or a word string to be recognized in speech recognition. The number given in the network is the ID number of the arc representing each word in the network. FIG. 16 shows the network storage unit 1 in FIG.
2 shows an example of a recognition network stored in the storage device 02. When it is possible to trace such a network from the beginning to the end, it is assumed that the input voice has been received by this network. For example, in the case of FIG. 16, it is possible to receive input voices such as "I want a ticket for a live tomorrow's Basia" and "I like a ticket for a concert in Van Hereen tomorrow". The word standard pattern storage unit stores in advance the voice standard pattern of the word to be recognized, and the word standard pattern storage unit 103 stores the standard pattern of words in the network. The standard pattern may be a combination of smaller units such as syllables, for example.

The continuous speech recognition unit 104 connects the standard patterns stored in the word standard pattern storage unit 103 in accordance with the word string generated by following the above-described network, and features the input speech received from the speech input unit 101. By performing pattern matching with a vector time series, a word string having the highest acoustic similarity to the input speech is selected. The continuous speech recognition unit 104 outputs an arc ID sequence of each word representing this word sequence to the intermediate expression generation unit 106 as a recognition result. An intermediate expression is created and output from the arc ID sequence input to the intermediate expression generation unit 106 and the storage contents of the semantic relationship storage unit 105.

Next, the processing of the intermediate expression generation unit 106 will be described. Suppose now that the input voice is "I want a ticket for a concert in Bahia for the moment." At this time, “1,3,5,6,10,11,13,15,1” are obtained as the speech recognition results from the arc ID numbers shown in FIG.
The sequence of the arc ID of 6, 17 "is
Passed to 6. The intermediate expression generation unit 106 generates an intermediate expression from the information in FIG. 3 and the ID string of the arc. First, Arc I
When the table of FIG. 3 is searched for the first one in the column of the D number, the parent ID number in the column of ID number 1 is 17, 18, 19
It is. Among them, the number 17 is in the arc ID string, so the parent of the intermediate expression is “want”, and it can be seen from the CASE column that the relation is “benefit”. Also, a feature “#noun semantic feature, 111” is added to “I” in the intermediate expression of this sentence from the feature information column. By repeating the above processing, an intermediate representation as shown in FIG. 17 is obtained.

Next, one embodiment of the second aspect of the present invention will be described. In one embodiment of the present invention, an intermediate expression is created by storing semantic information and feature information of words in a network for speech recognition and a sub-network called from each network in the semantic relation storage unit 105 in FIG. . FIG. 4A shows a main network used for speech recognition, and FIGS. 4B, 4C, and 4D show sub-networks called by the main network. For example, FIG.
(B) is a subnet “date and time” which is called at arc ID number 2 in the main net.

Now, it is assumed that the input sentence is "I want a ticket for a concert in Bercia that has just been done". Then, the arc number sequence of the speech recognition result is "1,2 (2), 3,4 (1), 5,6,8,1" according to FIGS.
0, 11, 12 (1) ". Here, for example," 4
“(1)” means that the first arc of the subnet is called from the fourth arc of the main net. In this case, the subnet of the artist name is determined from the sub-column of the row of ID number 4 in FIG. 6 (b), the CP is "Vasia" and the feature information is #
It can be seen that the noun semantic feature is 111. FIG.
Since the parent ID numbers are 11 and 12 in the row with the ID number 4 of “1” and “12 (1)” is in the ID column of the recognition result, FIG.
From (c), it indicates that the CP “wants” is indicated, and from the CASE column, “Basia” and “wants” are owned. By repeating the above processing, an intermediate representation as shown in FIG. 17 is obtained. Other processes are exactly the same as those of the first embodiment of the present invention. As a result, an intermediate expression can be created from the result of speech recognition using a network and a subnetwork called from the network.

Next, an embodiment of the third aspect of the present invention will be described. In one embodiment of the present invention, feature information to be added to another word is also stored in the semantic relationship storage unit 105 in FIG. 1, and an intermediate expression is created using this information.

For example, assume that the input voice is "I want one ticket". Then, the arc ID sequence of the voice recognition result is “1, 2, 4, 5, 8” in FIG. At this time, arc number 5 corresponds to ID number 5 in FIG.
From the row, it can be seen that the CP is “1” and its parent is “ticket” because “2” is in the arc ID column. In addition, it can be seen from the parent assignment information column that the feature information “#NUMBER, {SIN}” is added to the “ticket” to be the parent CP. As a result of such processing, FIG.
Can be created. Other processes are exactly the same as those of the first embodiment of the present invention.

Next, an embodiment of the present invention will be described. FIG. 2 is a configuration diagram showing another embodiment of the present invention. In the present embodiment, an intermediate expression is created by adding feature information to each word using not only the semantic relationship storage unit 205 but also the contents of the word dictionary storage unit 207. For example, since the feature information such as #noun semantic feature and #verb meaning classification is always the same for each word, the information of the word dictionary storage unit 207 can be used by using the information of the word dictionary storage unit 207 without describing in the semantic relationship storage unit 205. Assigned as feature information. Other processes are exactly the same as those of the first embodiment of the present invention.

Next, an embodiment of the present invention will be described.

For example, suppose that the input sentence is "I want a ticket for a concert of the upcoming Basia" in the network shown in FIG. Then, the column of arc ID numbers is “2, 4, 5, 9, 10, 12, 14, 15, 16”
Becomes It is also assumed that the contents of the semantic relationship storage unit are as shown in FIG. Here, it is assumed that the row whose ID number is D is used unconditionally for creating an intermediate expression regardless of the result of speech recognition. Since the ID number column of the line where the CP “I” is located is “D”, the contents of this line are also used for creating the intermediate expression. As a result, a correct intermediate representation as shown in FIG. 17 can be created. Other processes are exactly the same as those of the first embodiment of the present invention.

Next, an embodiment of the present invention will be described.

For example, in the network shown in FIG. 12, the input sentence is "I want a ticket for Basia." Then, the column of arc ID numbers is “1, 2, 4,
5, 7, 8 ". The contents of the semantic relationship storage unit are shown in FIG.
4 is assumed. Here, in the column called on-node, when the arc having the number described in this column is present in the speech recognition result, the contents of this line are used for creating an intermediate expression. At this time, since there is "2" in the arc ID number sequence, the contents of the line having the ID number "onl" are also used for creating the intermediate expression, and as a result, the correct intermediate expression as shown in FIG. Can be created. Other processes are exactly the same as those of the first embodiment of the present invention.

Next, an embodiment of the present invention will be described. 18 is a diagram illustrating an example of the contents of the semantic relationship storage unit for the network in FIG. 15, and FIG. 19 is a diagram illustrating an example of an intermediate representation of the network in FIG.

For example, in the network shown in FIG. 15, the input sentence is "I want a ticket for a concert in Basia". Then, the sequence of the arc ID numbers becomes “1, 2, 4, 5, 7, 8, 10, 11”. It is also assumed that the semantic relationship storage unit is as shown in FIG. Here, the off-node column is used for creating an intermediate expression when the arc of the number described in this column is present in the speech recognition result, even if the ID number in the same line is "D". Shall not be present. Now, the top row in FIG. 18 has the ID number "D", but has "1" in the off-node column, and has "1" in the arc ID number column of the speech recognition result. Information is not used. As a result, an intermediate representation as shown in FIG. 19 is obtained.

It is also assumed that the input sentence is "I want a ticket for a concert in Basia". Then, the column of arc ID numbers is "2, 4, 5, 7, 8, 10, 1" in FIG.
Since the ID number of the top row in FIG. 18 is “D”, the contents of this row are also used, and as a result, an intermediate representation as shown in FIG. 19 is obtained. As described above, in the present embodiment, an intermediate representation can be correctly created even when there are elements which are omitted or not in the network. It is.

[0047]

As described above, according to the continuous speech recognition method of the present invention, it is possible to recognize more natural and diverse sentences at the time of speech recognition and output the meaning expression. .

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing one embodiment of the present invention.

FIG. 2 is a configuration diagram showing another embodiment of the present invention.

FIG. 3 is a diagram showing an example of contents of a semantic relationship storage unit in FIG. 1;

FIG. 4 is a diagram illustrating a first example of a voice recognition network.

FIG. 5 is a diagram showing an example of contents of a semantic relationship storage unit for a main network in FIG. 4;

FIG. 6 is a diagram showing an example of contents of a semantic relationship storage unit for a sub-network in FIG. 4;

FIG. 7 is a diagram showing a second example of the voice recognition network.

FIG. 8 is a diagram showing an example of an intermediate representation of the network in FIG. 7;

9 is a diagram showing an example of the contents of a semantic relationship storage unit for a network in FIG. 7;

FIG. 10 is a diagram showing a third example of the voice recognition network.

FIG. 11 is a diagram showing an example of the contents of a semantic relationship storage unit for a network in FIG. 10;

FIG. 12 is a diagram illustrating a fourth example of the voice recognition network.

13 is a diagram showing an example of an intermediate representation of the network in FIG.

FIG. 14 is a diagram showing an example of the contents of a semantic relationship storage unit for a network in FIG. 12;

FIG. 15 is a diagram illustrating a fifth example of the voice recognition network.

FIG. 16 is a diagram illustrating a sixth example of the voice recognition network.

FIG. 17 is a diagram illustrating an example of an intermediate representation of the network in FIG. 16;

18 is a diagram showing an example of the contents of a semantic relationship storage unit for a network in FIG.

19 is a diagram illustrating an example of an intermediate representation of the network in FIG.

[Explanation of symbols]

 101, 201 voice input unit 102, 202 network storage unit 103, 203 word standard pattern storage unit 104, 204 continuous speech recognition unit 105, 205 semantic relationship storage unit 106, 206 intermediate expression generation unit 207 word dictionary storage unit

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-2-214930 (JP, A) JP-A-1-245297 (JP, A) JP-A-62-293472 (JP, A) Technology of IEICE Research report [voice] SP88-17 (June 1988) 45-52 (58) Field surveyed (Int. Cl. ⁶ , DB name) G10L 3/00 561 G10L 3/00 551 G06F 15/38 JICST file (JOIS)

Claims (7)

(57) [Claims] 1. A first storage means for storing a network relating to words representing a grammar to be subjected to continuous speech recognition, a continuous speech recognition means for recognizing continuous speech by combining standard patterns of words according to the network, A second storage unit for storing a semantic relationship between a certain word and another word in the network and feature information for each word; and a recognition result generated from the continuous speech recognition unit and the second storage unit. Output means for outputting an intermediate expression. 2. The continuous speech recognition means recognizes continuous speech by combining a predetermined standard pattern of a recognition unit according to the network and a sub-network called from each network. Memorize networks expressing the grammar to be recognized and sub-networks called from each network,
The output unit is configured to determine the intermediate between the recognition result generated from the continuous speech recognition unit, the semantic relation between a certain word and another word in the second storage unit, and the feature information for each word in the network and the sub-network. 2. The continuous speech recognition method according to claim 1, wherein an expression is output. 3. The apparatus according to claim 2, further comprising: a second storage unit that stores feature information to be added to another word when a certain word is recognized; and an adding unit that adds the information to the intermediate expression. 3. The continuous speech recognition method according to claim 1, wherein 4. The method according to claim 1, wherein when a certain word is recognized, feature information to be added to the word is extracted from a word dictionary storage unit, and the adding unit adds the information to the intermediate expression. 2. The continuous speech recognition method according to 2 or 3. 5. The word dictionary storage unit stores words that do not exist in a network expressing a grammar to be subjected to continuous speech recognition but are always necessary to create an intermediate expression, and the output unit uses the information based on the information. 5. The continuous speech recognition method according to claim 1, wherein an intermediate expression is created and output. 6. The word dictionary storage unit stores words to be supplemented when a specific word in a network expressing a grammar to be subjected to continuous speech recognition is recognized, and the output unit stores an intermediate word based on the information. 6. The continuous speech recognition method according to claim 1, wherein the expression is created and output. 7. The word dictionary storage unit stores a specific grammar of a continuous speech recognition target grammar to be deleted when a specific word in the network representing a grammar of the continuous voice recognition target is recognized. 7. The continuous speech recognition method according to claim 1, wherein a word is stored, and said output means creates and outputs an intermediate expression based on said information.