JP4829910B2 - Speech recognition error analysis apparatus, method, program, and recording medium therefor - Google Patents

Description

  The present invention relates to speech recognition technology. In particular, the present invention relates to a speech recognition error analyzing apparatus, method, program, and recording medium for analyzing the cause of a speech recognition error in a language model.

  When improving the acoustic model and the language model that constitute the speech recognition engine, it is efficient to improve from the part that easily causes a recognition error.

  In an acoustic model that determines which phoneme the input speech is close to, it is possible to identify a portion that is likely to cause a recognition error by creating a confusion matrix. The Confusion Matrix lists all phonemes that are likely to be confused with other phonemes. An acoustic model can be improved efficiently by creating a confusion matrix and identifying phonemes that are easily confused, and then making improvements from the phonemes that are easily confused.

  On the other hand, as a method for analyzing the performance of a language model, a method of evaluating a language model by perplexity is often used (for example, see Non-Patent Document 1). In speech recognition, recognition word candidates are narrowed down using word chain probabilities calculated by a language model. The perplexity is a value indicating the average number of branches from each word included in the recognized vocabulary to the next word, and indicates that the larger the value is, the more difficult it is to narrow down the recognized word candidates.

However, since the perplexity value does not clearly indicate which word string is difficult to narrow down candidates, it is not possible to identify a portion that is likely to cause a recognition error in the language model.
Lawrence Rabiner (Author), Biing-Hwang Juang (Author), Sadaaki Furui (Translation), "Basics of Speech Recognition (Lower)", NTT Advanced Technology Co., Ltd., 1995, p. 263-265

  As described above, the method for analyzing the performance of the language model described in Non-Patent Document 1 has a problem in that it is impossible to specify a portion that easily causes a recognition error in the language model.

  An object of the present invention is to provide a speech recognition error analysis apparatus, method, program, and recording medium for identifying a part that easily causes a recognition error in a language model.

  According to one aspect of the present invention, a speech recognition process is performed on a speech signal using a language model, and a word string (hereinafter referred to as a recognized word string) as a speech recognition result is assigned. A word string composed of one or a plurality of consecutive words that do not match the correct word string corresponding to the recognized word string in the recognized word string (hereinafter referred to as a recognized error word string), and the recognized error word A recognition error section composed of a string and one word before and after the string is extracted from the recognized word string. A starting error two-word set composed of the first word in the recognition error section and the first word in the recognition error word string is extracted. A starting correct two-word group composed of the first word in the recognition error section and the first word in the correct word string corresponding to the recognition error word string is extracted. Using the language model, the word chain probability of the starting part error two-word group and the word chain probability of the starting part correct word group are calculated. Comparing the word chain probability of the start part error two-word set with the word chain probability of the start part correct word pair, the start part correct two-word group having a word chain probability lower than the word chain probability of the start part error two-word set (Hereinafter referred to as a low-starting part correct two-word set) is extracted.

  The low start correct two-word set having a word chain probability lower than the word chain probability of the start-part error two-word set is a word string that causes a recognition error. Therefore, by extracting the low-start-part correct two-word set, it is possible to specify a portion that is likely to cause a recognition error in the language model.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
Recognition errors tend to occur continuously over several words, and the causes of recognition errors can be divided into two reasons: (1) the cause of recognition error start and (2) the cause of recognition error expansion. . The first embodiment specifies a portion that can cause a recognition error among the causes of the recognition error.

  An example of the first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a functional block diagram of an example of a speech recognition error analysis apparatus. FIG. 4 is a flowchart illustrating the processing flow of the speech recognition error analysis method.

  The speech recognition error analysis apparatus 1 according to the first embodiment includes a speech recognition unit 11, a recognition error section extraction unit 12, a start unit two-word set extraction unit 21, a start unit word chain probability calculation unit 22, and a solid line in FIG. The low start part correct answer two word set extraction part 23 is provided, for example.

<Step S1>
The speech recognition unit 11 performs speech recognition processing on the speech signal using an acoustic model, a language model, and a recognition dictionary, and assigns a word string that is a result of the speech recognition processing to the speech signal. Let the assigned word string be a recognized word string. Each word in the recognized word string is given a start time and an end time. The recognition word string is sent to the recognition error section extraction unit 12.
See, for example, Reference 1 for an overview of speech recognition processing.

[Reference 1] Hirokazu Masami, 5 others, “Free Speech Recognition Technology that Listens to Natural Conversations with Customers“ VoiceRex ””, NTT Technical Journal, November 2006, No. 18, vol. 11, p. 15-18
For example, the voice recognition unit 11 performs voice recognition processing on a voice signal including at least a sentence “Internet is not connected”, and the voice signal part “Internet is not connected” as shown by a solid line in FIG. Is assigned a recognition word string including the word string “(Internet) (win) (na) (ga) (rough) (no)”.

<Step S2>
The recognition error section extraction unit 12 compares the recognition word string with the correct word string corresponding to the recognition word string, and is composed of the recognition error word string, the recognition error word string, and one word before and after the recognition error word string. A recognition error interval is extracted.

  The recognition error word string is a word string composed of one or a plurality of consecutive words that do not match the correct word string corresponding to the recognition word string in the recognition word string. The extracted recognition error word string and recognition error section are sent to the start unit two-word set extraction unit 21.

  In the example shown in FIG. 5, the recognized word string and the correct word string corresponding to the recognized word string do not match in the four consecutive word parts (win) (na) (ga) (rough). Accordingly, (win) (na) (ga) (rough) is a recognition error word string. In addition, this recognition error word string includes the previous word (Internet) and the next word (not) (Internet) (win) (na) (ga) (rough) (no) It becomes a section.

  In general, a plurality of recognition error sections are extracted by the recognition error section extraction section 12 by the voice recognition processing of the voice recognition section 11. The following processing is performed for each of a plurality of recognition error intervals.

<Step S3>
As illustrated in FIG. 2, the start part two-word set extraction unit 21 includes a start part error two-word set extraction part 211 and a start part correct answer two-word set extraction part 212.

The start part error two word set extraction unit 211 extracts a start part error two word set from the recognition error word string and the recognition error section. The extracted start part error two-word set is sent to the start part word chain probability calculation part 22.
The start part error two-word set is a two-word composed of the first word in the recognition error section and the first word in the recognition error word string.

  In the example shown in FIG. 5, two words (Internet) (win) composed of the first word in the recognition error section (Internet) and the first word in the recognition error word string (win) are: It becomes a starting part error two word set.

<Step S4>
The start correct two-word set extraction unit 212 of the start two-word set extraction unit 21 extracts the start correct two-word set from the recognition error section and the correct word sequence corresponding to the recognition error word sequence. The extracted start unit correct answer two-word set is sent to the start unit word chain probability calculation unit 22.
The start part correct two-word group is a two-word composed of the first word in the recognition error section and the first word in the correct word string corresponding to the recognition error word string.

  In the example shown in FIG. 5, two words (Internet) (GA) composed of the first word (Internet) in the recognition error section and the first word (GA) in the correct word string corresponding to the recognition error word string. However, it becomes a starting part correct answer two word set.

<Step S5>
The start part word chain probability calculation part 22 calculates the word chain probability of the start part error two-word set and the word chain probability of the start part correct answer two-word set, respectively, using the same language model used by the speech recognition unit 11. To do. The calculated word chain probabilities are sent to the low start correct part two-word set extraction unit 23 together with the start part correct two-word set or the start correct part two-word set that is the basis of the calculation.

  The word chain probability is the probability of chaining from the first word of a two-word set to the second word of the two-word set, calculated using a language model (see, for example, Reference 2) .)

[Reference 2] Lawrence Rabiner (Author), Biing-Hwang Juang (Author), Sadaaki Furui (Translation), "Basics of Speech Recognition (Bottom)", NTT Advanced Technology Co., Ltd., 1995, p. 262-263
<Step S6>
The low start correct two-word set extraction unit 23 compares the word chain probability of the start correct two-word set with the word chain probability of the start correct two-word set, Also extract the correct two-word set of the starting part with a low word chain probability. A start correct two-word set having a word chain probability lower than the word chain probability of the start error two-word set is set as a low start correct two-word set.

  The low starting part correct two-word set may cause a recognition error to start because its word chain probability is lower than the word chain probability of the starting part error two-word set. Therefore, by extracting the low-start-part correct two-word set as described above, it is possible to specify a part that is likely to cause a recognition error in the language model. More specifically, it is possible to identify a portion that may cause a recognition error among the portions that are likely to cause a recognition error in the language model.

  The word chain probability that is assigned to the low-start-part correct two-word set that causes the start of recognition error is lower than the corresponding two-part start-part error two-word sets. It is thought that the reason is that an appropriate probability has not been learned because the start part correct answer two words do not appear or the number of appearances is small. Therefore, it is possible to improve by using the text in which the low start part correct two words frequently appear as language model learning data.

[Second Embodiment]
Hereinafter, an example of the second embodiment will be described. As mentioned above, recognition errors tend to occur continuously over several words. The causes of recognition errors are (1) the cause of recognition error start and (2) the cause of recognition error expansion. It can be divided into two. The second embodiment specifies both causes.

  Hereinafter, although the example of 2nd embodiment is demonstrated, only a different part from 1st embodiment is demonstrated, and duplication description is abbreviate | omitted about the part similar to 1st embodiment.

  The speech recognition error analysis apparatus according to the second embodiment includes, in addition to the components of the speech recognition error analysis apparatus 1 according to the first embodiment, an intra-section two-word set extraction unit 31 and intra-section word chain illustrated by broken lines in FIG. For example, a probability calculation unit 32 and a high interval error two-word set extraction unit 33 are provided. Further, in the speech recognition error analysis method of the second embodiment, in addition to the processes of the speech recognition error analysis apparatus 1 of the first embodiment, the processes of steps S7 to S10 illustrated by broken lines in FIG. 4 are performed.

<Step S2>
The recognition error section extraction unit 12 sends the extracted recognition error word string to the in-section two word set extraction unit 31. It is not necessary to send the recognition error section to the intra-section two-word set extraction unit 31.

<Step S7>
As illustrated in FIG. 3, the intra-section two-word set extraction unit 31 includes an intra-section error two-word set extraction unit 311 and a correct return two-word set extraction unit 312.

The intra-section error two-word set extraction unit 311 extracts all intra-section error two-word sets from the recognition error word string. The extracted intra-section error two-word sets are sent to the correct return two-word set extraction section 312 and the intra-section word chain probability calculation section 32.
The intra-section error two-word set is a set of two consecutive words in the recognition error word string.

  In the example shown in FIG. 5, (win) (na), (na) (ga), and (ga) (coarse) are the intra-section error two-word sets.

<Step S8>
The correct return two-word set extraction unit 312 of the intra-section two-word set extraction unit 31 extracts a correct return two-word set for each intra-section error two-word set from the intra-section error two-word set and the correct word string. The extracted correct answer return two-word group is sent to the intra-section word chain probability calculation unit 32.

  The correct return two-word set is the first word of the intra-interval error two-word set and the time after the beginning of the first word, and at the end of the first word in terms of time. It is a word string composed of words in a correct word string having the closest starting point.

  In the example shown in FIG. 5, the correct answer return two-word group corresponding to the intra-section error two-word group (win) (na) is (win) (connected). That is, the words in the correct word string that are temporally behind the beginning of the first word (win) in the intra-segment error two-word group (win) (na) are (connected) and (ra). It is. Of the (connected) and (ra), it is (connected) that has the start point closest in time to the end of (win). This is because the temporal distance between the end of (winning) and the starting end of (connected) is shorter than the temporal distance between the end of (winning) and the starting end of (ra). Therefore, the correct answer return two-word group corresponding to the intra-section error two-word group (win) (na) is (win) (connected). Similarly, the correct answer return two-word pair corresponding to the intra-section error two-word set (na) (ga) is (na) (ra), and the correct answer return corresponding to the intra-section error two-word set (ga) (rough) The two word set is (ga) (ra).

<Step S9>
The intra-section word chain probability calculation unit 32 calculates the word chain probability of the intra-section error two-word set and the word chain probability of the correct return two-word set using the same language model used by the speech recognition unit 11. . The calculated word chain probabilities are sent to the high-intersection error two-word group extraction unit 33 together with the intra-section error two-word group or the correct return two-word group that is the basis of the calculation.

<Step S10>
The high-interval error two-word set extraction unit 33 compares the word chain probability of the intra-section error two-word set with the corresponding word-chain probability of the correct return two-word set, and the word chain of the correct return two-word set An intra-section error two-word set having a word chain probability higher than the probability is extracted. An intra-section error two-word set having a word chain probability that is higher than the word chain probability of the correct return two-word set is defined as a high-section intra-error two-word set.

  The intra-high-interval error two-word group has a higher word chain probability than the word chain probability of the correct return two-word group, and can therefore cause recognition errors to expand. Therefore, by extracting the two-word group error in the high section as described above, it is possible to specify a part that is likely to cause a recognition error in the language model. More specifically, it is possible to identify a portion that can cause recognition errors in a portion that easily causes recognition errors in the language model.

  High word chain probabilities are assigned to the high-intra-error two-word pairs that cause recognition errors to expand. This is considered to be the cause. Therefore, it is possible to improve by selecting the text used for language model learning so as not to be biased to the two errors in the high section.

[Modification]
The start part appearance frequency totaling unit 24 indicated by a one-dot chain line in FIG. 1 may obtain the appearance frequency of the low start part correct two-word set (step S11, FIG. 4). For example, the number of each low start correct two-word set extracted by the low start correct two-word set extraction unit 23 is counted, and the count is assigned to each low start correct two-word set as an appearance frequency. Also, for example, the appearance frequency of the low start part correct two-word set = (the count number of the low start part correct two-word set) / (the sum of the count numbers of the low start part correct two-word set), and each low start part correct answer A ratio may be assigned as the appearance frequency to the two-word group.

  As described above, by providing the start part appearance frequency totaling unit 24, it is possible to extract a low start part correct answer two-word set having a high appearance frequency, and to select a low start part correct answer two word set to be improved. You can narrow down.

  Similarly, the intra-section appearance frequency totaling unit 34 indicated by the alternate long and short dash line in FIG. 1 may obtain the appearance frequency of the high-section intra-error two-word set (step S12, FIG. 4). As a result, it is possible to extract a high-frequency intra-error two-word set that has a high appearance frequency, and to narrow down high-error intra-word two-word sets to be improved.

  When the above-described configuration is realized by a computer, the processing contents of the functions of each unit of the speech recognition error analysis apparatus are described by a program. By executing this program on a computer, the functions of the above-described units are realized on the computer.

  That is, when the CPU sequentially reads and executes each program, the speech recognition unit 11, the recognition error section extraction unit 12, the start unit two word set extraction unit 21, the start unit word chain probability calculation unit 22, the low start unit correct answer 2 Word set extraction unit 23, start portion appearance frequency totaling unit 24, intra-section two-word set extraction unit 31, intra-section word chain probability calculation unit 32, high-section error two-word set extraction unit 33, and intra-section appearance frequency totaling section 34 Each function is realized. In this case, the CPU functioning as each unit of the speech recognition error device performs processing on data read from a recording medium such as a memory or a hard disk, and stores the processed data in the recording medium.

The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium may be any medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. Specifically, for example, the magnetic recording device may be a hard disk device or a flexible Discs, magnetic tapes, etc. as optical discs, DVD (Digital Versatile Disc), DVD-RAM (Random Access Memory), CD-ROM (Compact Disc Read Only Memory), CD
-R (Recordable) / RW (ReWritable), etc., MO (Magneto-Optical disc), etc. as a magneto-optical recording medium, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. as a semiconductor memory it can.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  As an execution form different from the above-described embodiment, the computer may read the program directly from the portable recording medium and execute processing according to the program. Each time is transferred, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to a computer but has a property that is based on computer processing).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

In addition, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. For example, in FIG. 4, the process of step S3 and the process of step S4 may be performed in parallel. Similarly, the process of step S7 and the process of step S8 may be performed in parallel. Further, the processing from step S3 to step S6 and the processing from step S7 to step S10 may be performed in parallel.
Needless to say, other modifications are possible without departing from the spirit of the present invention.

The functional block diagram of the example of a speech recognition error analysis apparatus. The functional block diagram of the example of a start part two word set extraction part. The functional block diagram of the example of the two word group extraction part in an area. The flowchart which illustrates the flow of a process of the speech recognition error analysis method. The figure showing examples, such as a recognition error word sequence, a recognition error section, a start part error two-word group, a start part correct answer two-word group, an intra-section error two-word group, and a correct return two-word group.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Speech recognition error analyzer 11 Speech recognition part 12 Error recognition section extraction part 21 Start part Two word set extraction part 22 Start part Word chain probability calculation part 23 Low start part Correct two word set extraction part 24 Start part appearance frequency totaling part 31 Intra-section two-word set extraction section 32 In-section word chain probability calculation section 33 In-high section error two-word set extraction section 34 In-section appearance frequency counting section 211 Start section Two-word set extraction section 212 Start section correct answer two-word set extraction section 311 Intra-section two-word set extraction unit 312 Correct answer return two-word set extraction unit

Claims (10)

A speech recognition unit that performs speech recognition processing on a speech signal using a language model and assigns a word string (hereinafter referred to as a recognition word string) that is a speech recognition result;
A word string composed of one or a plurality of consecutive words that do not match the correct word string corresponding to the recognized word string in the recognized word string (hereinafter referred to as a recognized error word string), and the recognized error word A recognition error interval extraction unit that extracts a recognition error interval composed of a sequence and one word before and after the sequence from the recognition word sequence;
A starter error two-word set extraction unit that extracts a starter error two-word set composed of a first word of the recognition error section and a first word of the recognition-error word string;
A starter correct two-word set extraction unit that extracts a starter correct two-word set composed of a first word of the recognition error section and a first word of a correct word string corresponding to the recognition-error word string;
Using the language model, a starter word chain probability calculating unit that calculates a word chain probability of the starter error two-word set and a word chain probability of the starter correct two-word set, respectively,
Comparing the word chain probability of the start part error two-word set with the word chain probability of the start part correct answer two-word set, the start part correct answer 2 having a word chain probability lower than the word chain probability of the start part error two-word set A low start part correct answer two-word set extraction unit that extracts a word set (hereinafter referred to as a low start part correct answer two word set);
A speech recognition error analysis apparatus comprising:   The speech recognition error analysis apparatus according to claim 1,
  Start part appearance frequency totalization part for obtaining the appearance frequency of the low start part correct two-word set,
  A speech recognition error analyzer further comprising:   In the speech recognition error analysis device according to claim 1 or 2,
  An intra-section error two-word set extraction unit that extracts all of a set of two consecutive words in the recognition-error word string (hereinafter referred to as an intra-section error two-word set) from the recognition-error word string;
  The first correct word in the intra-interval error two-word set and the correct word that is temporally after the start of the first word and has the start closest to the end of the first word in time A correct return two-word set extraction unit that extracts a correct return two-word set composed of the words in the sequence for each of the above-mentioned two error pairs in the section;
  Using the language model, an intra-segment word chain probability calculation unit for calculating a word chain probability of the intra-interval error two-word set and a word chain probability of the correct return two-word set, respectively,
  Compare the word chain probability of the two-word set in the interval and the word chain probability of the two-word set of correct return corresponding to the two-word set of errors in the interval, and more than the word chain probability of the two-word set of correct return A high-intersection error two-word set extraction unit that extracts an intra-section error two-word set with a high word chain probability (hereinafter referred to as a high-section error two-word set);
  A speech recognition error analyzer further comprising: In the speech recognition error analysis device according to claim 3 ,
Intra-section appearance frequency totaling unit for determining the appearance frequency of the above-mentioned high section error two-word set,
A speech recognition error analyzer further comprising: A voice recognition step in which a voice recognition unit performs voice recognition processing on a voice signal using a language model and assigns a word string (hereinafter referred to as a recognition word string) as a result of the voice recognition;
The recognition error section extraction unit is a word string composed of one or a plurality of continuous words that do not match the correct word string corresponding to the recognition word string in the recognition word string (hereinafter referred to as a recognition error word string). And a recognition error section extraction step for extracting a recognition error section composed of the recognition error word string and one word before and after the recognition error word string,
Start part error two-word set extraction unit extracts a start part error two-word set consisting of the first word of the recognition error section and the first word of the recognition error word string Steps,
Start of extracting the correct part two-word set of the start part composed of the first word in the recognition error section and the first word in the correct word string corresponding to the recognition error word string Part correct word two word set extraction step;
A starting word chain probability calculating step, wherein the starting word chain probability part calculates the word chain probability of the starting part error two-word set and the word chain probability of the starting part correct two-word set, respectively, using the language model; ,
The low start part correct two-word set extraction unit compares the word chain probability of the start part correct two-word set with the word chain probability of the start part correct two-word set, and the word chain probability of the start part error two-word set A low start part correct two-word set extraction step for extracting a start correct part two-word set (hereinafter referred to as a low start correct part two-word set) having a lower word chain probability,
A speech recognition error analysis method comprising:   The speech recognition error analysis method according to claim 5,
  A start part appearance frequency totaling step for obtaining the appearance frequency of the low start part correct two-word set,
  A speech recognition error analysis method further comprising:   The speech recognition error analysis method according to claim 5 or 6,
  The intra-section error two-word set extraction unit extracts all the consecutive two word sets (hereinafter referred to as intra-section error two-word sets) in the recognition error word string from the recognition error word string. A two-word set extraction step;
  The correct return two-word set extraction unit is temporally after the first word of the intra-section error two-word set and the first end of the first word, and at the end of the first word A correct return two-word set extraction step for extracting a correct return two-word set composed of a word in the correct word sequence having a starting point closest to
  An intra-segment word chain probability calculation unit, using the language model, to calculate the intra-interval error two-word group word chain probability and the correct answer return two-word group word chain probability, ,
  The high-intersection error two-word set extraction unit compares the word chain probability of the intra-section error two-word set with the word chain probability of the correct return two-word set corresponding to the intra-section error two-word set, and A high-intersection error two-word set extraction step for extracting an intra-section error two-word pair (hereinafter referred to as a high-section error two-word set) having a word chain probability higher than the word chain probability of the correct return two-word set;
  A speech recognition error analysis method further comprising: The speech recognition error analysis method according to claim 7 ,
Intra-section appearance frequency totaling section, the intra-section appearance frequency totaling step to obtain the appearance frequency of the above-mentioned high section error two-word set,
A speech recognition error analysis method further comprising:   5. A speech recognition error analysis program for causing a computer to function as each part of the speech recognition error analysis apparatus according to claim 1.   A computer-readable recording medium on which the speech recognition error analysis program according to claim 9 is recorded.

Images