JPH08123471A - Speech recognition device - Google Patents

Abstract

PURPOSE: To output a recognition result as a series of semantic origins through a backward search by extracting the semantic origins included in a spoken speech simultaneously with recognition. CONSTITUTION: A syntax network given semantic origins is generated from syntax and semantic knowledge wherein semantic origins are made to correspond to syntax rules and outputted to a syntax network storage part 3. A forward search part 5 searches a syntax hypothesis for an input speech according to the syntax network by using an acoustic dictionary part 4, and outputs a search history to a search history storage part 6. A search history rewriting part 7 refers to the semantic origins given to the syntax network and rewrites the search history held in the search history storage part 6. A backward search part 8 traces back the search history held in the search history storage part 6 to generate a recognition result as a series of semantic origins.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus used for a natural language man-machine interface.

[0002]

2. Description of the Related Art FIG. 25 shows, for example, Proceedings of 199.
1 International Conference on Acoustics, Speech &
It is a conventional continuous speech recognition apparatus shown on pages 701 to 704 of Signal Processing. Reference numeral 3 is a syntax network storage unit that holds a syntax network, 4 is an acoustic dictionary unit that holds a standard pattern of an acoustic model, and 5 is a syntax network that searches for a syntax hypothesis for an input speech using the syntax network and the acoustic model. Therefore, the syntax node of the syntax network, the time when the syntax node is reached, the search score at the time when the syntax node is reached, the syntax node which is one before the syntax node, and the syntax node which is one before the syntax node The time at which the syntax node and the 1
A forward search unit that outputs a search history including a word between the previous syntax nodes, 5 is a search history storage unit that holds the search history output from the forward search unit, and 8 is a search history that is stored in the search history storage unit. Is a backward search unit that reads out, follows the syntax network according to the search history, and generates a recognition result.

FIG. 26 is a diagram showing an input voice input to the forward search unit 5. The input voice is t ₀ ,
It is extracted in frame units at fixed time intervals such as t ₁ , t ₂ , .... Then, the extracted voice is subjected to frequency analysis, and, for example, 16-dimensional feature parameter v ₀ ,
v ₁ , v ₂ , ... Are extracted. The characteristic parameters v ₀ , v ₁ , v ₂ , ... Are input to the forward search unit 5.

FIG. 27 is a diagram showing the standard pattern of the acoustic model stored in the acoustic dictionary unit 4. For example, phoneme / a
HMM (Hidden Markov Model) as acoustic model of /
Pattern A by is registered. The forward search unit 5
The sequence of the input characteristic parameters v ₀ , v ₁ , v ₂ , ... Is shown in FIG. 27 as pattern A, pattern B, pattern C, ...
By performing comparison with, the likelihood calculation of each phoneme / a /, / i /, / u /, ... Is performed on the input feature parameter sequence.

FIG. 28 is a diagram showing an example of a syntax for generating a syntax network. The syntax rules are divided into a rule part and a dictionary part. What is described on the right side of the dictionary portion in FIG. 28 is called a terminal symbol. The terminal symbol is not expanded any further. That is, in the rule part and the dictionary part, the terminal symbol is not described on the left side. On the other hand, in the rule section and the dictionary section, the symbols sandwiched between <> are non-terminal symbols. The non-terminal symbol is described on the left side in the dictionary part. The rule part is described by non-terminal symbols on both the left side and the right side.

FIG. 29 is a diagram showing a syntax network created by the syntax shown in FIG. The syntax network shown in FIG. 29 is a network stored in the syntax network storage unit 3. In FIG. 29, N
1, N2, N3 ... Are syntax nodes. Also, 1
The arrow from one syntax node to another syntax node is called a syntax arc.

The forward search unit 5 receives, for example, the acoustically analyzed characteristic parameter v ₂₃ at a certain time t ₂₃ ,
By referring to the syntax network held in the syntax network storage unit 3 and the standard pattern held in the acoustic dictionary unit 4, for all syntax nodes of the syntax network,
The likelihood is calculated using the feature parameter and the standard pattern of words connected to the syntactic node. This likelihood is the search score. Then, for example, the search history as shown in FIG. 30 is output. The search history shown in FIG. 30 is the word hj in the syntax node N3 in the syntax network of FIG.
It shows the search history at time t ₂₃ for aku, g
n is the syntax node N3, frm is the syntax node N3 at time t ₂₃ , and prob is the syntax node N3 at time t ₂₃ .
_{At 23} , pgn arrives at the syntax node N2 that arrives one before the syntax node N3, sfrm arrives at the syntax node N2 that arrives one before, and the time t ₁₅ arrives, and the word immediately precedes the syntax node N3. Hj between the syntax nodes N2 reaching
aku is held.

Next, the operation of the backward search section 8 will be described with reference to FIGS. 31 and 32. As shown in FIG. 32, among the search histories generated at the final time t ₇₃ , the search history having the maximum search score is selected, and similarly, the search history having the maximum search score also at the time t ₃₅ . Select. In this way, the search history sequence (j)-(h)
-(F)-(c)-(a) is obtained. In this way, as one of the correct answer candidates, "go hjaku en deo"
We obtain a sequence of words "negaisimasu". Also, by combining the words of the second search score or the third search score, etc., the top N correct answer candidates are obtained and used as the result of speech recognition. Then, a process of extracting semantic features is performed on these output word sequences.

As described above, in the conventional voice recognition apparatus, all search histories output from the forward search section 5 are held in the search history storage section 6 and in the backward search section 8 in the search history storage section 6. The recognition result can be generated by reading the search history and following the search history on the syntax network.

[0010]

When the voice recognition device is used for a man-machine interface in natural language, for example, for a hotel reservation system by telephone, the meaning included in the recognition result to drive the system is determined. Need to be extracted. However, in the conventional speech recognition device as described above, since the recognition result output from the backward search unit 8 is a series of words, it is necessary to perform a process of extracting, for example, a semantic feature from the words after the recognition process. .

Further, since the search history storage unit 6 holds all the search histories output from the forward search unit 5, they are syntactically the same, but the particle hypotheses and endings are slightly different. Will be all retained. As a result, the top N recognition results output from the backward search unit 8 are occupied by semantically the same candidates, and correct answers are not included in the top N recognition results, so that a correct recognition result cannot be obtained. there were.

The present invention has been made to solve the above problems, and a first object of the present invention is to extract a semantic feature included in an utterance at the same time as recognition in a backward search and to obtain a semantic feature sequence. A voice recognition device for outputting a recognition result is obtained.

A second object of the present invention is to obtain a speech recognition apparatus which outputs a large number of correct answer candidates which are semantically different from each other without the top N recognition results being occupied by the semantically same candidates.

[0014]

A speech recognition apparatus according to the present invention includes a syntactic network storage unit for holding a syntactic network to which semantic information is added, an acoustic dictionary unit for holding a standard pattern of an acoustic model, and the syntactic network. Using the acoustic model, a search for a syntactic hypothesis for input speech is performed according to the syntax network, and a forward search unit that outputs a search history and a search history storage unit that holds the search history output from the forward search unit A search history rewriting unit that rewrites the search history held in the search history storage unit with reference to the semantic information given to the syntax network; and a search history stored in the search history storage unit, A backward search unit that generates a recognition result by tracing the syntax network according to the search history is provided. Characterized in that was.

The speech recognition apparatus further includes a syntax / semantic knowledge storage unit that holds syntax / semantic knowledge in which semantic information is associated with syntactic knowledge that defines a grammar of input speech, and the syntax / semantic knowledge storage unit.
It is characterized by comprising a syntax network generation unit for generating a syntax network to which semantic information is added from semantic knowledge.

A speech recognition apparatus according to the present invention includes a syntactic network storage unit that holds a syntactic network to which semantic information and operation rules are added, an acoustic dictionary unit that holds a standard pattern of an acoustic model, the syntactic network and the acoustics. Using the model, a search for a syntactic hypothesis for input speech is performed according to the syntax network, a forward search unit that outputs a search history, a search history storage unit that holds the search history output from the forward search unit, and A search history rewriting unit that rewrites the search history held in the search history storage unit by referring to the semantic information given to the syntax network, and a search history stored in the search history storage unit are read out, and stored in the search history. Therefore, the syntactic network is traced, and the semantic information and the operation rule given on the syntactic network are Ri performs calculation of semantic information, and further comprising a backward search unit for outputting a recognition result.

In the speech recognition device, further, in the syntactic knowledge that defines the grammar of the input speech, the dictionary part that defines the word associates the word with the semantic information, and the rule part associates the operation rule of the semantic information. It is characterized by further comprising a syntax / semantic knowledge storage unit that holds the syntax / semantic knowledge, and a syntax network generation unit that generates a syntax network to which semantic information and operation rules are added from the syntax / semantic knowledge.

In the speech recognition apparatus according to the present invention, the forward search unit receives a syntax node of the syntax network, a time when the syntax node reaches the syntax node, a search score at the time at the syntax node, and one before the syntax node. The syntactic node that has reached, the time when the syntactic node that has reached the preceding syntax node is reached, and the word between the syntactic node and the syntax node that is the previous syntax node are output as a search history, and the search history rewriting unit When a plurality of search histories for the same time and same syntax node held in the history storage unit have the same word semantic information, a part of the search history having the same semantic information is searched. The history is left in the search history storage unit, and another search history having the same semantic information is deleted from the search history storage unit.

The search history rewriting unit is a search history stored in the search history storage unit, and a search history in which the semantic information of a word is specific is the syntax that arrives immediately before the search history is stored. It is characterized in that it is rewritten with a search history corresponding to the time when the node and the syntax node which arrived one before are reached.

[0020]

In the speech recognition apparatus configured as described above, since the search history rewriting unit rewrites the search history held in the search history storage unit by referring to the semantic information given to the syntax network, At the same time as the recognition in the search unit, the meaning included in the utterance is extracted, and the recognition result as a meaning series can be output.

Further, since the syntax / semantic knowledge storage unit holds the semantic information in association with the syntactic knowledge, the syntactic network generation unit automatically generates the syntactic network to which the semantic information is added.

Furthermore, since the backward search unit calculates the semantic information according to the semantic information and the arithmetic rule of the semantic information provided on the syntax network, the arithmetic result of the semantic information can be output as the recognition result. .

Further, since the syntax / semantic knowledge storage unit holds the semantic information and the operation rule, the syntax network generation unit automatically generates the syntax network to which the semantic information and the operation rule are added.

Further, the search history rewriting unit leaves a part of the search history stored in the search history storage unit and having the same word semantic information at the same time and same syntax node in the search history storage unit, Since other search histories are deleted from the search history storage unit, the number of search histories held in the search history storage unit that have the same semantic hypothesis at the same time and same syntax node decreases, and the backward search unit The top N recognition results to be output are
This will include many correct answer candidates that are semantically different.

Furthermore, the search history rewriting unit holds the search history stored in the search history storage unit, the search history having a particular semantic feature of a word.
The search history corresponding to the time at which the previous syntax node and the time at which the previous syntax node was reached are rewritten, so words with specific semantic features are deleted from the syntax hypothesis, and the backward search section outputs. The top N recognition results that are set include many correct answer candidates that are semantically different.

[0026]

【Example】

Example 1. FIG. 1 shows a speech recognition apparatus according to an embodiment of the present invention. Reference numeral 1 is a syntax / semantic knowledge holding syntax / semantic knowledge in which semantic information is associated with syntactic knowledge defining a grammar of input speech. A storage unit 2, a syntax network generation unit that generates a syntax network to which semantic information is added from the syntax / semantic knowledge, and a syntax network storage unit 3 that holds the syntax network generated by the syntax network generation unit. Reference numeral 4 denotes an acoustic dictionary unit that holds a standard pattern of an acoustic model, and 5 uses the syntax network and the acoustic model to search for a syntax hypothesis for an input speech according to the syntax network. The time at which the syntax node is reached, the search score at the time at the syntax node, the syntax node at which the syntax node has arrived one before the syntax node, the time at which the syntax node has reached the previous syntax node, the syntax node and the It is a forward search unit that outputs a search history including a word between previous syntax nodes. 6 is a search history storage unit that holds the search history output from the forward search unit, and 7 is a search for rewriting the search history stored in the search history storage unit by referring to the semantic information given to the syntax network. It is a history rewriting unit. Reference numeral 8 denotes a backward search unit that reads out the search history held in the search history storage unit and follows the syntax network according to the search history to generate a recognition result.

FIG. 2 shows an example of the syntax / semantic knowledge stored in the syntax / semantic knowledge storage unit 1. For example,
Rule <hundred>: = hja whose right side is terminal symbol hjaku
In ku, the semantic feature 100 is used as the semantic information and the terminal symbol h
It is defined for jaku.

FIG. 3 shows an example of the syntax network generated by the syntax network generation unit 2 from the syntax / semantic knowledge stored in the syntax / semantic knowledge storage unit 1 and stored in the syntax network storage unit 3. is there. Each syntactic arc is given a terminal symbol of syntactic knowledge, and at the same time, a semantic feature for the terminal symbol. For example, in the syntax arc of the terminal symbol hjaku, the terminal symbol hjaku
A semantic feature 100 is given as the semantic information for the.

In FIG. 2 and FIG. 3, the semantic feature NULL is given as the semantic information for the case particle and the ending. The reason why NULL is given to the case particle and the ending is that it is considered to have no significant meaning in the speech recognition processing. For example, when asking the amount of money in the hotel reservation system, the voice recognition device may recognize the amount of money as an answer. Therefore, if the answer is "Please give me 500 yen", "I like 500 yen", or "I have 500 yen", etc., the voice recognition device should be able to recognize "500 yen". , No need to give meaning to other words. Therefore, as shown in FIGS. 2 and 3, the case particle and the ending are given a semantic feature of NULL.

Also in this embodiment, the forward search unit 5 outputs the search history as shown in FIG. 4, as in the conventional example. The search history rewriting unit 7 refers to the syntax network held in the syntax network storage unit 3 to rewrite the word written in the word column of the search history to the semantic feature of the word. That is, hjaku is rewritten to 100. FIG. 5 shows an example in which the search history of FIG. 4 is rewritten.

The operation of the backward search unit 8 will be described when the search history shown in FIG. 6 is held in the search history storage unit 6. First, the search history indicated by the search history (j) having the maximum search score among the search history corresponding to the final syntax node N6 in the syntax network at the final time t ₇₃ of the input voice is stored in the search history memory. Select from Part 6. next,
The value gn has a value equal to the value N5 of pgn in the search history (j), and f has a value equal to the value t ₃₅ of sfrm in the search history (j).
Of the search histories (g), (h), and (i) held in rm, the search history (h) having the largest search score is selected from the search history storage unit 6. Thereafter, the search history is traced in the same manner to obtain the search history sequence (j)-(h)-(f)-(c)-(a). Since the obtained search history sequence is reverse in time, it is sequentially searched from the last search history (a) in the sequence.
By referring to the rd column, the sequence of semantic features “5 100
"MONEY NULL NULL" is output as the recognition result.

As described above, the speech recognition apparatus according to this embodiment has a syntax / semantic knowledge storage unit for storing syntax / semantic knowledge, in which semantic information is associated with syntactic knowledge defining the grammar of input speech, A syntax network generation unit that generates a syntax network to which semantic information is added from the syntax / semantic knowledge, a syntax network storage unit that holds the syntax network generated by the syntax network generation unit, and a standard pattern of an acoustic model. Using the acoustic dictionary unit, the syntax network and the acoustic model, a search for a syntax hypothesis for input speech is performed according to the syntax network, the syntax node of the syntax network, the time when the syntax node is reached, and the syntax node. The search score at the time, the syntax node that has reached the previous syntax node, the one , A forward search unit for outputting a search history including a time at which the syntax node has been reached, a word between the syntax node and the previous syntax node, and a search for holding the search history output from the forward search unit. A history storage unit, a search history rewriting unit that rewrites the search history stored in the search history storage unit by referring to the semantic information given to the syntax network, and a search history stored in the search history storage unit. A backward search unit for generating a recognition result by reading out and tracing on the syntax network according to the search history is provided.

Example 2. The operation of the search history rewriting unit 7 of FIG. 1, which is an embodiment of the present invention, will be described. In the second embodiment, the operations of 1 to 6 and 8 in FIG.
Since it is the same as the first embodiment, the description thereof will be omitted. In this embodiment, an example in which a large number of semantically similar correct answer candidates are prevented from occurring will be described. For example, the recognition result is "go hjak
u ”or“ go bjaku ”or
In any case of "go pjaku", "5
An example in which these three recognition results are combined into one recognition result will be described.

FIG. 7 shows an example of three search histories among the search histories stored in the search history storage unit 6, where gn, frm, pgn, and sfrm are all the same. The search history rewriting unit 7 obtains the semantic features of the words described in the word column of each search history by referring to the syntax network held in the network storage unit 3. Since all the words written in the three search histories shown in FIG. 7 have the same semantic feature 100, the search history rewriting unit 7 causes the search history prob whose search score prob is not the maximum (b).
And the search history (c) is deleted from the search history storage unit 6.
This is nothing but dismissed semantically identical two syntaxes hypotheses reaching the syntax node N3 at time t _23. Therefore, the backward search unit 8 does not generate a recognition result based on these two syntax hypotheses.

FIG. 8 is a timing chart showing the operation of the search history rewriting section 7. Search histories (a) to (l) shown in FIG. 6 are generated by the forward search unit 5 at the time shown in FIG.
Is stored. For example, at time t ₂₃ , the search history (c), (d), (e) is once stored in the search history storage unit 6. The search history rewriting unit 7 inspects the three search histories generated at time t ₂₃ , and as described above, gn, f
If rm, pgn, and sfrm are all equal, remove the one with the highest search score and delete the others. In the example shown in FIG. 8, the search history (c) is left and the search history (d) and (e) are deleted. The same is done at time t ₃₅ , only the search history (h) is left, and the other search histories (g) and (i) are deleted. Furthermore, at time t ₇₃ , the search history (g) is left and the other search histories (k) and (l) are deleted.

FIG. 9 is a diagram showing a state before the search history is deleted and a state after the search history is deleted. FIG. 9A is a diagram showing a recognition result when the search history rewriting unit has the semantic feature adding function according to the first embodiment. FIG. 9B shows the recognition result when the search history rewriting unit has the search history deleting function according to the second embodiment. In the case of FIG. 9A, 2 × 3 × 3 × 3 = 54 combinations can be considered, and the upper N correct answer candidates will be selected from the 54 candidates. However, among the 54 combinations, there are many that have the same semantic features.
As a result, the same number of items are included in the top N items. However, as shown in FIG. 9 (b), when the same meaning is deleted, there are only two combinations, and many correct answer candidates having different meanings can be obtained for the top N. It will be possible.

In FIG. 8, it is explained that the search history rewriting unit 7 deletes the search history at each time when the search history is newly generated. Is deleted at a time when is stored, and the search history rewriting unit 7 deletes the unnecessary search history stored in the search history storage unit immediately before the backward search unit 8 executes the backward search. It doesn't matter.
Note that FIG. 9C will be described in Example 3 described later.

As described above, in this embodiment, when there are a plurality of search histories held at the same time and same syntax node in the search history storage unit, which have the same semantic feature of a word, the meaning is Rewriting a search history that leaves only the search history having the same maximum score in the search history storage unit having the same feature in the search history storage unit and deletes another search history having the same semantic feature from the search history storage unit It has a section.

Example 3. The operation of the search history rewriting unit 7 of FIG.
The case where the search history shown in FIG. 10 is held will be described. In the third embodiment, the operations of 1 to 6 and 8 in FIG. In this embodiment, a case will be described in which a semantic feature that does not affect the recognition result is deleted in advance.
For example, when there is a recognition result of "500 yen", this is set to "500 yen". If the semantic feature of "de" is NULL, "de"
Considering that it may not be necessary, the case of deleting this "by" will be described below.

The search history rewriting unit 7 has a certain semantic feature in the search history stored in the search history storage unit 6,
For example, the following rewriting is performed on the search history (d) of a word having a semantic feature NULL that is not necessary as a recognition result. First, a search history (a) having a value in gn that is equal to the value N4 of pgn in the search history (d) and having a value in frm that is equal to the value t ₃₀ of sfrm in the search history (d),
(B) and (c) are selected from the search history storage unit 6. Next, the search score 182.395 of the search history (d) and the maximum search score of the search history (a), (b), and (c) are 1
The difference with 58.962 is delta = 23.433. Then, the search history (d) is deleted from the search history storage unit 6, and instead, as shown in FIG. 11, the gn value N5 and the frm value of the search history (d) are the search history (d). The value of frm t ₃₅ and the value of prob in d) are deleted to the values of prob in the search history (a), (b), and (c), respectively.
The sum of a, the value of sfrm is the value of sfrm in search history (a), (b), and (c), and the value of word is the value of search history (a), (b), and (c), respectively.
Create search histories (e), (f), (g) that are values of
Write to the search history storage unit 6.

FIG. 12 is a diagram showing the state of the search history storage unit 6. FIG. 12A shows the search history storage unit shown in FIG. Further, FIG. 12B shows the state of the search history storage unit shown in FIG. In addition, FIG.
[Fig. 8] is a diagram showing a recognition result. FIG. 13A is a diagram of FIG.
The recognition results shown in are shown. FIG. 13B is a diagram shown in FIG.
The recognition results shown in are shown.

FIG. 9C shows a case in which the search history of words having a semantic feature NULL, which is not necessary as a recognition result according to the third embodiment, is deleted from the results recognized in the first and second embodiments. It is a figure which shows the state of. Figure 9
As shown in (c), “MO between syntax nodes N3 to N6
Two "NULL" are deleted and "MONY NULL" will be recognized as "MONEY".

FIG. 14 is a diagram showing another example in which the deletion of the search history according to the second and third embodiments is further combined. When the search history as shown in FIG. 14A is stored, the unnecessary search history as shown in the third embodiment is rewritten (the search history whose semantic feature is NULL), so that FIG. The state becomes as shown in. By deleting the search history having the same semantic feature as shown in the second embodiment in the state shown in FIG. 14B, the state shown in FIG. 14C is obtained. As described above, by combining the second embodiment and the third embodiment, it is possible to reduce the search history that is not necessary as the recognition result or the search history having the same meaning. In the example shown in FIG. 9C, the second embodiment is applied first, and then the third embodiment is applied. In the case of FIG. 14, Example 3 is applied first, and then Example 2 is applied. By applying any of the embodiments first, the number of search histories can be effectively reduced. Thus, for example, applying Example 2 and then applying Example 3 and then applying Example 2 again, performing both cases where either Example 2 or Example 3 is applied first. It is desirable to do.

As described above, in this embodiment, in the search history stored in the search history storage unit, the semantic features of words are N
A search history rewriting unit that rewrites a search history, which is a specific thing called ULL, with a search history corresponding to the syntax node that has reached the previous syntax node and the time that the syntax node has reached the previous syntax node held by the search history. It is equipped with.

Example 4. 1 is an embodiment of the present invention,
The syntax / semantic knowledge storage unit 1, the syntax network generation unit 2, the syntax network storage unit 3, and the backward search unit 8 in FIG. 1 will be described. In addition, in Example 4, 4 of FIG.
Since the operation of No. 7 is the same as that of the first embodiment, the description thereof will be omitted. In the above-described embodiment, for example, a recognition result of "5 100" can be obtained, but the actual meaning is 5 x 100.
It is desirable to obtain a recognition result of = 500. In this embodiment, a case where a calculation such as 5 × 100 is performed and the result can be output as a recognition result will be described.

FIG. 15 shows an example of the syntax / semantic knowledge stored in the syntax / semantic knowledge storage unit 1. In the dictionary unit that defines the words, the words are associated with the semantic features, and in the rule unit, the semantic features are calculated. It is the correspondence between the rules. For example, sem (<Equation 1>) represents a semantic feature obtained from the non-terminal symbol <Equation 1>. Further, the semantic feature calculation rule sem (<price 2>) = sem (<expression 1>) × sem (<expression 2>) defined in the syntax rule (2) of FIG.
The semantic feature for the non-terminal symbol <Charge 2> is sem
It is obtained from the product of (<Equation 1>) and sem (<Equation 2>).

FIG. 16 shows the syntax network generation unit 2 based on the syntax / semantic knowledge stored in the syntax / semantic knowledge storage unit 1.
3 shows an example of a syntax network generated by the syntax network storage unit 3 and held in the syntax network storage unit 3. Each syntactic arc is given a terminal symbol of syntactic knowledge, and at the same time, a semantic feature for the terminal symbol. For example, in the syntax arc of the terminal symbol hjaku, the terminal symbol hjak
A semantic feature of 100 is added to u. In addition, for example, the syntax node N1 has a semantic feature calculation rule sem.
(<Charge 2>) = sem (<Numerical expression 1>) × sem (<Numerical expression 2>) and sem (<Numerical expression 1>) = sem (go>) are given.

The operation of the backward search unit 8 will be described when the search history storage unit 6 holds the search history shown in FIG. From the search history storage unit 6, the search history (c) having the maximum search score is selected from the search times corresponding to the final time t ₂₃ of the input voice and the final syntax node N3 in the syntax network. Then, the semantic feature calculation is performed as follows. First, the word hja that is the value of word
The semantic feature sem (hjaku) = 100 of ku is obtained by referring to the syntax network. Then pgn
The semantic feature calculation sem (<hundred>) = sem (hjaku) sem (<numerical formula 2>) = sem (<hundred>) with reference to the semantic feature calculation rule assigned to the syntax node N2 that is the value of To obtain sem (<Equation 2>) = 100. When the semantic feature calculation is completed, gn has a value equal to the pgn value N2 of the search history (c), and the sfrm value t of the search history (c).
A search history (a) having a value equal to ₁₅ in frm is selected from the search history storage unit 6. Then, the semantic feature calculation is performed as follows. First, the semantic feature sem (go) = 5 of the word go, which is the value of word, is obtained by referring to the syntax network. Next, the semantic feature is calculated with reference to the semantic feature calculation rule assigned to the syntax node N1 that is the value of pgn, and sem (<Equation 1>) = 5 is obtained. Further, by the operation of the semantic feature in the syntax node N2, s
Since em (<Equation 2>) = 100 is obtained, sem
(<Charge 2>) = sem (<Equation 1>) × sem (<Equation 2>) = 5 × 100 = 500 is obtained.

The backward search section 8 uses the syntax node N of FIG.
Semantic feature calculation rule sem (sentence) = se given to 1
From m (<Charge 2>), a semantic feature of 500 is obtained as a recognition result.
Is output.

As described above, in this embodiment, in the syntactic knowledge that defines the grammar of the input speech, the dictionary part that defines the word associates the word with the semantic feature, and the rule part associates the semantic feature calculation rule. The syntactic / semantic knowledge storage unit that holds the syntactic / semantic knowledge, and the search history stored in the search history storage unit are read, traced on the syntactic network according to the search history, and the semantic features assigned on the syntactic network. And a backward search unit that performs a semantic feature calculation according to the semantic feature calculation rule and outputs a semantic feature sequence as a recognition result.

Embodiment 5 FIG. In this embodiment, a case where continuous speech recognition using frame synchronization is based on the N-Best paradigm will be described. In addition, in this example, a case where the method used in Examples 1 to 4 described above is compared with a conventional lattice N-Best method in particular will be described. In continuous speech recognition, as an efficient search algorithm based on the N-Best paradigm, R.
Schwartz and Y.-L. Chow: ■ The N-Best algorithm:
An efficent and exact procedure for finding the N
most likely sentence hypotheses ■, Proc. ICASSP, p
p.81-84 (1990). and R. Schwartz and S. Austin: ■ A com
parison of several approximate algorithms for find
ing multiple (N-BEST) sentence hypotheses ■, Proc.
ICASSP, pp.701-704 (1991). Is proposed. In this approach, many syntactic hypotheses that are semantically the same with slightly different particles and endings are generated during the forward search, and the top N correct answer candidates that are obtained include many semantically identical candidates. As a result, the correct candidates are not included in the top N pieces, and a correct recognition result cannot be obtained. Therefore, in the fifth embodiment, the above-described first embodiment is used.
As in the case of ~ 4, in order to obtain many correct answer candidates that are semantically different for a small N, a method of pruning the same hypothesis during the forward search and extracting the semantic features included in the utterance at the same time as recognition To propose.

The purpose of speech recognition in the system is to obtain a semantic feature sequence as an input parameter to the system from the user's utterance. Therefore, it is important to obtain more semantically different correct answer candidates from the limited N correct answer candidates so that the correct feature semantic sequence is not lost from the recognition result. Minami, Yamada, Yoshioka, Kano: "2-stage LR parser considering meaning in free speech recognition" as a method of using meaning during recognition, sound structure theory,
pp. 69-70 (1993) proposes a beam search using meaning in the framework of HMM-LR.
In the forward search of the frame-synchronous N-Best search, the present method prunes hypotheses that are semantically the same using the semantic information given to the syntactic network.

As shown in FIG. 18, the syntax used in the recognition unit of the dialogue system defines the highest-level rule (rule having a sentence start symbol on the left side) as a series of items designating the operation of the system. Each item is defined by the in-item grammar, and the dictionary part of the grammar defines the phoneme sequence for the non-terminal symbol and the semantic feature for the word. In the rule part of the grammar, a calculation rule for semantic features is defined. The calculation rule is used to generate one semantic feature from a plurality of semantic features during traceback. In the example of FIG. 18,
The semantic feature of <charge 3> is obtained as the product of the semantic feature $ 1 for the non-terminal symbol <number> and the semantic feature $ 2 for <thousand>. As one of the semantic features defined for a word, a semantic feature “NULL” having no meaning is defined.
For example, two utterances of "August 26th" and "August 26th" represent the same semantic feature, and the case particle "no" has no semantic feature. For such words, "NUL
L ”is given. The semantic information described above is embedded in the syntax arc and the syntax node when the syntax is expanded in the syntax network.

R. Schwartz and S. Austin: ■ A compari
son of several approximate algorithms for finding
multiple (N-BEST) sentence hypotheses ■, Proc. ICA
SSP, pp.701-704 (1991) .lattice N-Bes.
In the t method, the traceback pointers for all the words that enter the syntax node are held, but in the method proposed in this embodiment, the hypothesis pruning is performed by the two methods described below. (1) Traceback pointers having the same start frame time and transition source syntax node and the same semantic feature leave only the one with the maximum score (see FIG. 19). (2) The traceback pointer of the word whose semantic feature is "NULL" is rewritten using the traceback pointer of the start time of the word held in the transition source syntax node (see FIG. 20).

According to (1), the semantic features are equal and time t '
The plurality of hypotheses exiting node S ′ at time t and entering node S at time t are reduced to one. Further, according to (2), the word whose semantic feature is "NULL" is deleted from the hypothesis (see FIG. 21). Further, as shown in FIG. 22, hypotheses having the same semantic feature even if the time is different in the middle of the syntax nodes, only the one having the maximum score is left and all other hypotheses are deleted from the hypothesis. After deleting "NULL" by
Apply (1).

The method according to this embodiment was evaluated by an unspecified speaker / continuous speech recognition experiment whose task was hotel reservation. The syntax rules used have a relatively high degree of freedom, including unnecessary words, and have 286 word syntax rules and 146 words. As the input sentence, a total of 125 sentences in which 5 adult men uttered 25 kinds of sentences were used. The number N of correct candidates output as a recognition result was changed to 5, 10, 20, ..., 100, and it was examined how many candidates having different sequences of semantic features were included. FIG. 23 shows the average of the experimental results of 125 sentences. Also,
FIG. 24 shows the recognition rate in the semantic feature series. Conventional lat
In the nice N-Best method, there are 4.9 candidates with different semantic feature sequences among the top 100 correct candidates, and it can be said that most of the candidates have the same meaning. On the other hand, in the method proposed in this embodiment, 5.1 candidates having different semantic feature sequences exist up to the 10th position, and l
Compared with the attic N-Best method, many correct answer candidates that are semantically different with a small N are obtained. Also in terms of recognition rate, the results up to the 10th place in this method are latt
It is better than the result up to the 100th position in the ice N-Best method. However, even after applying this method, the N candidates do not all have different meanings. This is "NULL" in the forward search.
This is because it is not possible to prune hypotheses with different start times that have semantic features other than. On the other hand, by pruning candidates having the same semantic feature sequence during backward traceback, more semantically different candidates can be obtained.

As described above, in this embodiment, in the forward search, the hypotheses that are semantically the same are pruned, the search is efficiently performed, and many correct answer candidates having different meanings are obtained for small N. Proposed. The result of the recognition experiment, la
In the tissue N-Best method, candidates of the semantic feature series that were not obtained even when considering up to the 100th position were obtained by traceback up to the 10th position, recognizing the effectiveness of this method.

[0058]

Since the present invention is constructed as described above, it has the following effects.

By providing the search history rewriting section for rewriting the search history held in the search history storage section by referring to the semantic information given to the syntax network, the meaning included in the utterance at the same time as recognition in the backward search section Can be extracted and the recognition result as a sequence of meanings can be output.

Further, it is possible to automatically generate a syntax network to which semantic information is added.

Further, by providing a backward search for calculating the semantic information according to the semantic information and the semantic information arithmetic rule given on the syntactic network and outputting the sequence of the semantic information as a recognition result, utterance is recognized and uttered at the same time. The semantic information contained in can be extracted.

Further, it is possible to automatically generate a syntactic network to which semantic information and operation rules are added.

A search history rewriting unit for deleting an unnecessary search history from the search history storage unit, which has the same word semantic information at the same time and same syntax node, is deleted from the search history storage unit. By providing, the number of search histories held in the search history section and having the same syntax hypothesis at the same time and same syntax node is reduced, and as the top N recognition results output by the backward search section. , Which includes many correct answer candidates that are semantically different.

Further, in the search history stored in the search history storage unit, the search history whose semantic information of a word is specific is the syntax node and the previous syntax node that has reached the previous one held by the search history. By providing the search history rewriting unit that rewrites with the search history corresponding to the time when the syntax node has reached, the word whose semantic information is specific is deleted from the syntax hypothesis, and the top N output by the backward search unit is output. As a result of recognition of the individual, one including many correct answer candidates that are semantically different is obtained.

[Brief description of drawings]

FIG. 1 is a functional block configuration diagram of a voice recognition device showing an embodiment of the present invention.

FIG. 2 is a diagram showing an example of syntax / semantic knowledge held in a syntax / semantic knowledge storage unit according to the first embodiment of the present invention.

FIG. 3 is a diagram showing an example of a syntax network generated by a syntax network generation unit from the syntax / semantic knowledge held in the syntax / semantic knowledge storage unit of the present invention and held in the syntax network storage unit.

FIG. 4 is a diagram showing an example of a search history according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an example of a search history rewritten by a search history rewriting unit according to the first embodiment of the present invention.

FIG. 6 is a diagram showing an example of a search history stored in a search history storage unit according to the first embodiment of the present invention.

FIG. 7 shows gn, frm, in Example 2 of the present invention.
FIG. 6 is a diagram showing an example of a search history in which pgn and sfrm are all the same, and words have the same semantic features.

FIG. 8 is a diagram showing an operation of a search history rewriting unit in the embodiment of the present invention.

FIG. 9 is a diagram showing an operation of a search history rewriting unit in the embodiment of the present invention.

FIG. 10 is a diagram showing an example of a search history according to the third embodiment of the present invention.

FIG. 11 is a diagram showing an example of a search history newly created by a search history rewriting unit according to the third embodiment of the present invention.

FIG. 12 is a diagram showing an operation of a search history rewriting unit in Embodiment 3 of the present invention.

FIG. 13 is a diagram showing an operation of a search history rewriting unit in Embodiment 3 of the present invention.

FIG. 14 is a diagram showing an operation of a search history rewriting unit in Embodiments 2 and 3 of the present invention.

FIG. 15 is a diagram showing an example of syntax / semantic knowledge held in a syntax / semantic knowledge storage unit according to the fourth embodiment of the present invention.

FIG. 16 is a diagram showing an example of a syntax network generated by a syntax network generation unit from the syntax / semantic knowledge stored in the syntax / semantic knowledge storage unit according to the fourth embodiment of the present invention and stored in the syntax network storage unit. .

FIG. 17 is a diagram showing an example of a search history stored in a search history storage unit according to the fourth embodiment of the present invention.

FIG. 18 is a diagram showing an example of syntax in Example 5 of the present invention.

FIG. 19 is a diagram showing pruning in semantic feature units in a forward search in Example 5 of the present invention.

FIG. 20 is a semantic feature NUL according to the fourth embodiment of the present invention.
The figure which shows the rewriting of the trace back pointer with respect to L.

FIG. 21 shows that the semantic feature in the fifth embodiment of the present invention is NU.
The figure which shows the deletion of the word which is LL.

FIG. 22 is a diagram showing pruning of a hypothesis with a different time at a node on the way in the fifth embodiment of the present invention.

FIG. 23 is a diagram showing an experimental result of the present invention.

FIG. 24 is a diagram showing a recognition rate in the semantic feature series of the present invention.

FIG. 25 is a diagram showing a conventional voice recognition device.

FIG. 26 is a diagram showing an input voice.

FIG. 27 is a diagram showing an audio dictionary unit.

FIG. 28 is a diagram showing a conventional syntax.

FIG. 29 is a diagram showing a conventional syntax network.

FIG. 30 is a diagram showing a conventional search history.

FIG. 31 is a view for explaining the operation of a conventional voice recognition device.

FIG. 32 is a view for explaining the operation of a conventional voice recognition device.

[Explanation of symbols]

1 syntax / semantic knowledge storage unit, 2 syntax network generation unit, 3 syntax network storage unit, 4 acoustic dictionary unit, 5
Forward search unit, 6 Search history storage unit, 7 Search history rewriting unit, 8 Reverse search unit.

Claims (6)

[Claims] 1. A syntactic network storage unit that holds a syntactic network to which semantic information is added, an acoustic dictionary unit that holds a standard pattern of an acoustic model, and a syntactic hypothesis for input speech using the syntactic network and the acoustic model. Is performed according to the syntax network to output a search history, a forward search unit, a search history storage unit that holds the search history output from the forward search unit, and the semantic information given to the syntax network is referred to. Then, a search history rewriting unit that rewrites the search history stored in the search history storage unit, and a search history stored in the search history storage unit are read,
A voice recognition device comprising a backward search unit for generating a recognition result by tracing on the syntax network according to the search history. 2. The speech recognition device further comprises a syntax / syntax in which semantic information is associated with syntactic knowledge defining a grammar of input speech.
The speech recognition apparatus according to claim 1, further comprising: a syntax / semantic knowledge storage unit that holds semantic knowledge, and a syntax network generation unit that generates a syntax network to which semantic information is added from the syntax / semantic knowledge. 3. A syntactic network storage unit that holds a syntactic network to which semantic information and operation rules are added, an acoustic dictionary unit that holds a standard pattern of an acoustic model, and an input speech using the syntactic network and the acoustic model. A forward search unit that performs a search of a syntactic hypothesis according to the syntax network, and outputs a search history, a search history storage unit that holds the search history output from the forward search unit, and a meaning given to the syntax network. A search history rewriting unit that rewrites the search history stored in the search history storage unit with reference to information; and a search history stored in the search history storage unit,
Tracing the syntax network according to the search history,
A speech recognition apparatus comprising a backward search section that calculates the semantic information according to the semantic information and the arithmetic rule provided on the syntax network and outputs a recognition result. 4. The voice recognition device further associates semantic information with a word in a dictionary unit that defines a word in syntactic knowledge that defines a grammar of an input voice, and calculates a calculation rule of the semantic information in a rule unit. A syntactic / semantic knowledge storage unit that holds the syntactic / semantic knowledge associated with each other, and a syntactic network generating unit that generates a syntactic network to which semantic information and operation rules are added from the syntactic / semantic knowledge. The voice recognition device according to claim 3. 5. The forward search unit, the syntax node of the syntax network, the time when the syntax node is reached, the search score at the time in the syntax node, the syntax node that arrived one before the syntax node, The time when the syntax node reached the previous one, the syntax node and the 1
The word between the preceding syntax nodes is output as a search history, and the search history rewriting unit is a search history for the same time and same syntax node held in the search history storage unit, and the word semantic information is the same. When there is a plurality of certain ones, some search histories among the search histories having the same semantic information are left in the search history storage unit, and other search histories having the same semantic information are used as the search histories. The voice recognition device according to claim 1, wherein the voice recognition device is deleted from the storage unit. 6. The search history rewriting unit holds, in the search history stored in the search history storage unit, a search history in which the semantic information of a word is specific 1
The speech recognition apparatus according to claim 1, wherein the speech recognition device is rewritten with a search history corresponding to the time when the syntax node reached the previous time and the time when the syntax node reached the previous time was reached.