JP2010204274A - Speech recognition device and method and program therefore - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech recognition device capable of obtaining a recognition result with a small amount of calculation. <P>SOLUTION: The speech recognition device includes: a feature extracting section 111 for extracting a sound feature amount for each frame from input speech; a search section 112 which performs speech recognition by searching and pruning the sound feature amount on compression network 102 which is created by merging a plurality of adjoining nodes in a search network 101, and by searching and pruning them until an end point of the input speech, by removing a node in the search network 101 corresponding to the pruned node in the compression network 102. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a speech recognition apparatus, a method thereof, and a program thereof.

  Conventionally, a technique has been proposed in which clustering is performed on a word-by-word basis, matching processing is performed on the word with the largest initial score (representative word) in the cluster, and then other words in the cluster are reevaluated. (See Patent Document 1).

  In speech recognition, if the number of recognized vocabulary is large, the search network becomes large, and a large amount of calculation is required for the search. As a method for reducing the calculation amount of the search, a beam search as shown in Non-Patent Document 1 is used. In the beam search, the search space is narrowed down and the amount of calculation is reduced by alternately repeating the search for the nodes on the search network and the branch hunting at each step of the search. However, since the search network has many branches near the beginning of a sentence or near a word boundary, the number of nodes to be searched increases even when beam search is used, and a large amount of calculation is required.

  On the other hand, Patent Document 2 proposes the following three methods.

  In the first method, in the first search, a search is performed using a search network that has been reduced by merging similar phonemes in the vicinity of the beginning of the word. Next, when the search result is not uniquely determined by phoneme merging, a search network is configured without merging phonemes from the first search result, and the amount of calculation is reduced by performing the second search.

  In the second method, only a recognition candidate narrowed down with a small amount of calculation using a rough standard pattern is re-matched using a precise standard pattern.

  The third method does not use a precise standard pattern as compared to the second method, and does not perform rematching.

X. Huang, A.A. Acero and H.M. -W. Hon, “Spoken Language Processing,” Prentice Hall PTR, pp. 606-608, 2001.

JP 2005-134442 A JP 2001-31293 A

  However, each of the above prior arts has a problem in that the amount of calculation for speech recognition is not sufficiently reduced, and when it is reduced, recognition accuracy deteriorates.

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide a speech recognition apparatus, a method thereof, and a program thereof that can obtain a highly accurate speech recognition result with a small amount of calculation. And

  The present invention provides a feature extraction unit that extracts an acoustic feature amount from an input speech for each frame, and the acoustic feature amount on at least one compression network generated by merging a plurality of adjacent nodes in a search network. Search and branch hunting, exclude nodes of the search network corresponding to the branch-pruned nodes of the compression network from search targets, perform search and branch hunting to the end of the input speech, And a search unit for recognizing the speech recognition apparatus.

  According to the present invention, a highly accurate speech recognition result can be obtained with a small amount of calculation.

  Hereinafter, a speech recognition apparatus according to an embodiment of the present invention will be described with reference to the accompanying drawings.

(First embodiment)
A speech recognition apparatus according to a first embodiment will be described with reference to FIGS.

  The configuration of the speech recognition apparatus of this embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a speech recognition apparatus according to this embodiment.

  The speech recognition apparatus includes a feature extraction unit 111, a search network 101, a compression network 102, and a search unit 112.

  The feature extraction unit 111 extracts an acoustic feature amount for each frame from the input voice.

  The search network 101 receives an acoustic feature amount from the feature extraction unit 111.

  The compressed network 102 is generated by merging adjacent nodes with high similarity in the search network 101.

  The search unit 112 performs a search using the search network 101 and the compression network 102 and outputs a recognition result.

  Note that this voice recognition device can also be realized, for example, by using a general-purpose computer as basic hardware. At this time, the voice recognition apparatus may be realized by installing the above program in a computer in advance, or may be stored in a storage medium such as a CD-ROM or distributed through the network. This program may be realized by appropriately installing it on a computer.

  First, the search network 101 and a search method using the search network 101 will be described with reference to FIG.

  In FIG. 11, each phoneme is modeled by a three-state left-to-right HMM, and the recognition target words are “Sato”, “Shibata”, “Hirai”, “Honda”, “Kitamura”, “Katayama”, “Takahashi” The example of the search network 101 in the recognition of the isolated word which is nine words of "Nakai" and "Maki" is shown. A black circle represents a start node, a white circle represents a normal node, a double circle represents an end node, and a word in the square is a word label attached to an arc connected by a dotted line.

  Various search methods have been proposed. In the present embodiment, a method called “token passing” will be described.

  In token passing, a token is assigned to a node, and every time an input frame is given, the token is propagated along the arc to the next node. This token holds a cumulative score and a history of word labels on the path through which the token has been propagated.

  First, a token is assigned to the start node before the search starts. Given an input frame, propagate the token along the arc to the next node. Here, when a plurality of tokens are propagated to one node, the token having the best cumulative score is selected. Then, the score of the node where the token exists is evaluated and added to the cumulative score of each token. Finally, the history of the word label held in the token assigned to the terminal node at the end of voice input is output as the recognition result.

  When performing an N-best search, when a plurality of tokens are propagated to one node, instead of selecting the token with the best cumulative score, the top N tokens (word label history and Corresponding cumulative score).

  In the search on the compression network 102 described later, it is not necessary to output the recognition result. This is because a recognition result is obtained by searching on the search network 101. Therefore, it is not necessary to maintain a history of word labels in the search on the compression network 102, and a special management such as management of a plurality of tokens (cumulative scores corresponding to the history of word labels) in N-best recognition. There is no need to process.

  Next, a method for generating the compression network 102 will be described. As the generation method, there are the following methods.

  The first method is a method of merging adjacent nodes whose similarity is greater than a predetermined value. Note that a node adjacent to a certain node is typically a sibling node having the same parent node or child node. However, it may include a parent node and a child node that are directly connected by an arc.

  The second method is a method of repeatedly merging adjacent nodes having the highest degree of similarity until the compression rate given by the following equation (1) becomes larger than a predetermined value.

Compression rate = number of network nodes after compression / number of network nodes before compression (1)

However, it is not restricted to these methods.

  The operation of the speech recognition apparatus will be described with reference to FIGS. FIG. 2 is a flowchart showing the operation when one compression network 102 is used in the speech recognition apparatus.

  In step S <b> 1111, the feature extraction unit 111 performs feature extraction for each frame at a certain time interval from the input speech to obtain an acoustic feature amount.

  In step S <b> 1121, the search unit 112 first performs a search on the compressed network 102 using the acoustic feature amount obtained by the feature extraction unit 111.

In step S <b> 1122, the search unit 112 performs branch hunting on the compression network 102.

  In step S <b> 1123, the search unit 112 performs a search on the search network 101. At this time, the search unit 112 reduces the amount of calculation for the search by excluding the nodes corresponding to the nodes that are pruned on the compressed network 102 from the search target.

  In step S <b> 1124, the search unit 112 performs branch hunting on the search network 101.

  In step S1125, the search unit 112 repeats the above steps (in the case of N) and outputs a recognition result (in the case of Y) when reaching the end of the input speech.

  The above steps S1121 to S1125 are search steps performed by the search unit 112. Next, the operation of steps S1121 to S1123 will be described with reference to FIGS. 3 and 4 in comparison with the operation of the conventional speech recognition apparatus.

  FIG. 3 is an example of the search network 101. Circles represent network nodes, numbers in circles represent node numbers, arrows represent arcs, and “word 1” to “word 6” represent word labels attached to the arcs.

  FIG. 4 is an example of the compressed network 102 generated by merging adjacent nodes having a high degree of similarity in the search network 101 shown in FIG. The compressed network 102 is generated by merging the nodes 1 and 2 of the search network 101 with the node-1 and the nodes 3, 4, and 5 of the search network 101 with the node-2. Further, the node-0 and the node-3 correspond to the node 0 and the node 6 of the search network 101, respectively, and are not merged, so that they are equivalent to the node 0 and the node 6, respectively.

  In the following description, a case where a search is performed from the node 0 of the search network 101 and the corresponding node 0 of the compression network 102 by the conventional speech recognition apparatus and the speech recognition apparatus of this embodiment will be described.

  In the conventional speech recognition apparatus, the search is performed only on the search network 101 without using the compression network 102. Therefore, when searching from node 0, it is necessary to search for 6 nodes from node 1 to node 6. Here, the search includes a likelihood calculation for an acoustic feature amount at each node, a cumulative likelihood calculation until reaching each node, and management of a word history.

  Next, the speech recognition apparatus of this embodiment will be described with reference to FIGS. 2, 3, and 4. FIG.

  In step S1121, the search unit 112 performs a search on the compressed network 102 in FIG. When the search is performed from the node-0, the search is performed on the three nodes, the node-1, the node-2, and the node-3. Here, since no word label is given to the arc of the compression network 102, it is not necessary to manage the word history in the search on the compression network 102. Further, in the N-best search, it is necessary to manage the top N word histories, but in the search of the compression network 102, since the word history is not managed, it is necessary to perform special processing also in the N-best search. Absent.

  In step S1122, the search unit 112 performs branch hunting on the node-1, the node-2, and the node-3 that have been searched in step S1121.

  In step S <b> 1123, the search unit 112 searches only for nodes in the search network 101 corresponding to nodes that have not been pruned in the compressed network 102.

  For example, in step S1122, when the search unit 112 does not prune node-2 and node-3 and does not prune node-1, the search unit 112 corresponds to node-2 in step S1123. The node 6 corresponding to the node 3 to the node 5 and the node 3 is excluded from the search targets, and only the node 1 and the node 2 corresponding to the node-1 are searched. At this time, a search is performed for a total of five nodes, that is, three nodes in the compression network 102 and two nodes in the search network 101.

  As a result, the speech recognition apparatus according to the present embodiment can reduce the number of search nodes as compared with the conventional speech recognition apparatus, and can obtain a recognition result with a small amount of calculation.

  In addition, when the search unit 112 does not branch the node-1 and the node-2 and does not branch the node-3 in step S1122, the search unit 112 determines that the node-1 and the node-2 in step S1123. The nodes 1 to 5 corresponding to are excluded from the search targets, and only the node 6 corresponding to the node-3 is searched. At this time, a total of four nodes, that is, three nodes in the compression network 102 and one node in the search network 101 are searched.

  As a result, the speech recognition apparatus according to the present embodiment can reduce the number of search nodes as compared with the conventional speech recognition apparatus, and can obtain a recognition result with a small amount of calculation. In addition, since the node-3 of the compressed network 102 and the node 6 of the search network 101 are equivalent, the likelihood for the acoustic feature amount in the node 6 is the same as the likelihood in the node-3, and it is not necessary to newly calculate again. Therefore, the calculation amount can be further reduced.

  Furthermore, when node-1 and node-3 are pruned in step S1122, and node-2 is not pruned, in step S1123, the search unit 112 selects node 1, node 2 and node 1 corresponding to node-1. The node 6 corresponding to the node-3 is excluded from the search targets, and only the node 3 to the node 5 are searched. At this time, a total of 6 nodes are searched, 3 nodes in the compression network 102 and 3 nodes in the search network 101. In this case, the total number of search nodes is the same as that of a normal speech recognition apparatus, but word history management is not performed in the search on the compression network 102. Therefore, even if the total number of search nodes is the same, the amount of calculation can be reduced as compared with a normal speech recognition apparatus.

  According to the speech recognition apparatus of the present embodiment, search and branch hunting are performed on the compression network 102, and nodes of the search network 101 corresponding to nodes that have been branch hunted on the compression network 102 are excluded from search targets. The amount of calculation can be reduced.

  In addition, since each frame is searched not only on the compression network 102 but also on the search network 101, but also on the search network 101, the recognition result and the calculation on the search network 101 are performed immediately after the end of the utterance. The cumulative likelihood for the input speech to be obtained can be obtained.

(Example of change)
A plurality of compression networks 102 can be used.

  In this case, for example, the first compression network 102 and the (k + 1) th compression network 102 are used. However, k = 1,..., K−1.

  The first compression network 102 is a compression network 102 generated by merging the search network 101 and adjacent nodes of the search network 101 having high similarity. The (k + 1) th compression network 102 is generated by merging the search network 101 and the adjacent nodes of the kth compression network 102 that have high similarity.

  Next, search and branch hunting are performed on the Kth compression network 102, and the nodes of the kth compression network 102 corresponding to the nodes hunted on the k + 1th compression network 102 are excluded from the search targets. Next, a search is performed on the kth compression network 102.

  These branch hunting and searching are repeated from k = K−1 to k = 1.

  Finally, a search is performed on the search network 101 by excluding the nodes of the search network 101 corresponding to the nodes hunted on the first compression network 102 from the search target.

(Second Embodiment)
A speech recognition apparatus according to a second embodiment will be described with reference to FIGS.

  The configuration of the speech recognition apparatus of this embodiment will be described with reference to FIG. FIG. 5 is a block diagram showing the speech recognition apparatus.

  The speech recognition apparatus includes a feature extraction unit 211, a search network 201, a partial compression network 202, and a search unit 212.

  The feature extraction unit 211 extracts an acoustic feature amount for each frame from the input voice.

  The partial compression network 202 is generated by merging adjacent nodes having a high similarity with the partial network of the search network 201 when an acoustic feature amount is input from the feature extraction unit 211.

  The search unit 212 performs search processing using the search network 201 and the partial compression network 202, and outputs a recognition result.

  Next, a method for selecting a partial network of the search network 201 will be described.

  In the first selection method, a partial network whose branch number is larger than a predetermined value is selected.

  In the second selection method, a partial network whose compression rate given by equation (1) is larger than a predetermined value is selected.

  However, it is not restricted to these methods.

  Next, a method for generating the partial compression network 202 will be described.

  The first generation method is a method of merging large nodes whose similarity is adjacent to a predetermined value.

  The second generation method is a method in which merging of adjacent nodes having the highest similarity is repeated until the compression ratio given by equation (1) becomes larger than a predetermined value.

  However, it is not restricted to these methods.

  The entire search network 201 is also included in the partial network, and the compressed network generated for the entire search network 201 is also included in the partial compressed network 202.

  The operation of the speech recognition apparatus will be described with reference to FIGS. Here, FIG. 6 is a flowchart showing the operation of the speech recognition apparatus.

  In step S <b> 2111, the feature extraction unit 211 performs feature extraction for each frame at a fixed time interval from the input speech to obtain an acoustic feature amount.

  In step S <b> 2121, the search unit 212 first determines whether or not the partial compression network 202 corresponding to the search portion of the search network 201 exists using the acoustic feature amount obtained by the feature extraction unit 211. If the partial compression network 202 exists, the process proceeds to step S2122 (in the case of Y), and if not, the process proceeds to step S2124 (in the case of N).

  In step S2122, the search unit 212 performs a search on the partial compression network 202, and performs branch hunting in step S2123.

  In step S2124, the search unit 212 performs a search on the search network 201. At this time, if there is a partial compression network 202 corresponding to the partial network of the search network 201 included in the search target, the node corresponding to the branch-hunted node on the partial compression network 202 is excluded from the search target. . This reduces the amount of calculation required for the search.

  In step S <b> 2125, the search unit 212 performs branch hunting on the search network 201.

  In step S2126, the search unit 212 repeats the above steps (in the case of N), and outputs a recognition result when it reaches the end of the input speech (in the case of Y).

  According to the speech recognition apparatus of this embodiment, search and branch hunting are performed in the partial compression network 202. Next, the nodes of the search network 201 corresponding to the nodes that are pruned by the partial compression network 202 are excluded from the search targets. Thereby, the increase in the calculation amount in the partial network can be suppressed, and the calculation amount can be reduced more efficiently.

(Example of change)
In the present embodiment, a plurality of partial compression networks 202 may be generated and used for each partial network, as in the first embodiment.

  Further, the partial compression network 202 may be generated and used for the partial network of the partial compression network 202.

(Third embodiment)
A speech recognition apparatus according to a third embodiment will be described with reference to FIGS.

  The configuration of the speech recognition apparatus of this embodiment will be described with reference to FIG. FIG. 7 is a block diagram showing the speech recognition apparatus according to the present embodiment.

  The speech recognition apparatus includes a feature extraction unit 311, a search network 301, a compressed network generation unit 312, and a search unit 313.

  In the feature extraction unit 311, the speech recognition apparatus extracts an acoustic feature amount for each frame from the input speech.

  The compressed network generation unit 312 generates a partial compressed network 302 by merging adjacent nodes with high similarity to the partial network of the search network 301.

  When the acoustic feature amount is input from the feature extraction unit 311, the search unit 313 performs search processing using the search network 301 and the partial compression network 302 and outputs a recognition result.

  The operation of the speech recognition apparatus of this embodiment will be described with reference to FIGS. FIG. 8 is a flowchart showing the operation of the speech recognition apparatus.

  In step S <b> 3121, the compressed network generation unit 312 generates the partial compressed network 302 by merging adjacent nodes with high similarity to the partial network of the search network 301. Note that the method for selecting a partial network in the search network 304 is the same as in the second embodiment. The method for generating the partial compression network 302 is the same as in the second embodiment.

  In step S <b> 3111, the feature extraction unit 311 performs feature extraction for each frame at a certain time interval from the input speech to obtain an acoustic feature amount.

  In step S <b> 3131, the search unit 313 first determines whether the partial compression network 102 corresponding to the search portion of the search network 101 exists using the acoustic feature amount obtained by the feature extraction unit 311. When it exists, it progresses to step S3132, and when it does not exist, it progresses to step S3134.

  In step S3132, since the partial compression network 102 exists, the search unit 313 searches on the partial compression network 102, and in step S3133 performs branch hunting.

  In step S3134, the search unit 313 performs a search on the search network 101. At this time, if the partial compression network 102 corresponding to the partial network of the search network 101 included in the search target exists, the node corresponding to the node that has been pruned on the partial compression network 102 is excluded from the search target. . This reduces the amount of calculation required for the search.

  In step S <b> 3135, the search unit 313 performs branch hunting on the search network 101.

  In step S3136, the search unit 313 repeats the above steps (in the case of N), and outputs a recognition result when it reaches the end of the input speech (in the case of Y).

  According to the speech recognition apparatus of this embodiment, the partial compression network 102 corresponding to the partial network of the search network 101 is generated without generating the compression network 102 in advance. Then, search and branch hunting are performed in the partial compression network 102, and then the nodes of the search network 101 corresponding to the nodes that have been branch hunted in the partial compression network 102 are excluded from search targets. Thereby, the amount of calculation can be reduced.

(Fourth embodiment)
A voice recognition device according to a fourth embodiment will be described with reference to FIGS. 9 and 10. This speech recognition apparatus is applied to continuous speech recognition.

  The configuration of the speech recognition apparatus of this embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing the speech recognition apparatus according to the present embodiment.

  The speech recognition apparatus includes a feature extraction unit 411, an acoustic model 401, a word dictionary 402, a language model 403, a search network generation unit 412, a compression network generation unit 413, and a search unit 414.

  The feature extraction unit 411 extracts an acoustic feature amount for each frame from the input voice.

  The search network generation unit 412 generates a search network 101 by developing word hypotheses from the acoustic model 401, the word dictionary 402, and the language model 403.

  The compressed network generation unit 413 generates a partial compressed network 102 by merging adjacent nodes having a high degree of similarity with the generated partial network of the search network 101.

  When an acoustic feature amount is input from the feature extraction unit 411, the search unit 414 performs search processing using the search network 101 and the partial compression network 102, and outputs a recognition result.

  The operation of the speech recognition apparatus according to this embodiment will be described with reference to FIGS. FIG. 10 is a flowchart showing the operation of the speech recognition apparatus according to this embodiment.

  In step S <b> 4111, the feature extraction unit 411 performs feature extraction for each frame at a certain time interval from the input speech to obtain an acoustic feature amount.

  In step S <b> 4121, the search network generation unit 412 generates a search network 404 by developing word hypotheses according to the intermediate results of the search using the acoustic model 401, the word dictionary 402, and the language model 403.

  In step S4131, the compressed network generation unit 413 generates a partial compressed network 405 by merging adjacent nodes having a high degree of similarity with the partial network of the generated search network 404. Note that the method for selecting a partial network in the search network 404 is the same as in the second embodiment. The method for generating the partial compression network 405 is the same as that in the second embodiment.

  The following steps S4141 to S4146 are the same as steps S2121 to S2126 of FIG. 6 in the second embodiment.

  In conventional continuous speech recognition, a large number of word hypotheses are developed at word boundaries, and a search network including a partial network having a very large number of branches is created. Therefore, it is necessary to search a large number of nodes.

  However, even in such a case, according to the speech recognition apparatus of the present embodiment, the partial compression network 405 corresponding to the generated partial network of the search network 404 is generated to perform search and branch hunting, and the partial compression network By excluding the node of the search network 404 corresponding to the node hunted in 405 from the search target, the amount of calculation can be reduced.

(Example of change)
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  For example, in each of the above embodiments, in order to create the compression network 102, the adjacent nodes to be merged are defined as child nodes having a common parent node. Child nodes may be merged.

  As a result, “child node having a common parent node” compresses in the space (vocabulary) direction, whereas when the parent node and child node are merged, it can be compressed in the time direction. For example, one word with different vowel lengths, such as “Uncle / o-jis-a-n” and “Grandfather / o-j-i-s-a-n”. Can be summarized.

It is a block diagram which shows the structure of the speech recognition apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of 1st Embodiment. It is an example of a search network. It is an example of the compression network 102 produced | generated from the search network of FIG. It is a block diagram which shows the structure of the speech recognition apparatus which concerns on 2nd Embodiment. It is a flowchart which shows operation | movement of 2nd Embodiment. It is a block diagram which shows the structure of the speech recognition apparatus which concerns on 3rd Embodiment. It is a flowchart which shows operation | movement of 3rd Embodiment. It is a block diagram which shows the structure of the speech recognition apparatus which concerns on 4th Embodiment. It is a flowchart which shows operation | movement of 4th Embodiment. It is explanatory drawing of the search network of 1st Embodiment.

101 ... Search network 102 ... Compression network 111 ... Feature extraction unit 112 ... Search unit

Claims (7)

A feature extraction unit that extracts an acoustic feature amount from the input speech for each frame;
Search and branch hunting is performed for the acoustic feature on at least one compression network generated by merging a plurality of adjacent nodes in the search network, and corresponds to the branch-hunted node of the compression network The search network node to be excluded from search targets, search and branch hunting to the end of the input speech, and a speech recognition search unit,
A speech recognition apparatus comprising: The compressed network is generated with a partial network that is part of the search network.
The speech recognition apparatus according to claim 1. The compressed network is a network that is generated by merging a number of branches from one parent node in the search network larger than an arbitrary number.
The speech recognition apparatus according to claim 1. The compression network is a network generated by merging child nodes of the search network and having a compression ratio larger than an arbitrary value.
The speech recognition apparatus according to claim 1. A plurality of adjacent nodes for merging refers to a parent node in the search network and a child node connected to the parent node,
The speech recognition apparatus according to claim 1. A feature extraction unit for extracting an acoustic feature amount for each frame from the input speech; and
A search unit searches and branches the acoustic feature on at least one compression network generated by merging a plurality of adjacent nodes in the search network, and the branch of the compression network is picked. A search step for recognizing speech by excluding a node of the search network corresponding to the selected node from a search target, performing search and branch hunting to the end of the input speech,
A speech recognition method comprising: On the computer,
A feature extraction function that extracts acoustic features from the input speech for each frame;
Search and branch hunting is performed for the acoustic feature on at least one compression network generated by merging a plurality of adjacent nodes in the search network, and corresponds to the branch-hunted node of the compression network A search function for recognizing speech by excluding nodes of the search network to be searched and performing search and branch hunting to the end of the input speech;
A speech recognition program for realizing

Images