JP2016099515A - Voice recognition error correction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice recognition error correction device capable of more accurately correcting a voice recognition error even if a similar expression is included in a document text.SOLUTION: A voice recognition error correction device 800 includes a preprocessing unit 811. The preprocessing unit 811 calculates distance representing similarity between a plurality of sequences about sequences of language elements included in the document text stored in a document text storage part, and detects similar sequences as to whether the calculated distance falls below a prescribed threshold. When the similar sequences are detected, warning information is outputted, and a correction proposal for solving similarity by operating the document test about at least any of the detected similar sequences is outputted. Also, the document text is corrected according to an instruction inputted from the outside. The document text preprocessed in this way is used to construct a finite state transducer.SELECTED DRAWING: Figure 14

Description

  The present invention relates to a speech recognition error correction apparatus.

In recent years, research has been conducted on techniques for automatically providing subtitle texts in programs such as television.
Non-Patent Document 1 describes an example of Spanish CARTV as one of the approaches for providing captions to a live broadcast news program. The system described in Non-Patent Document 1 is based on the premise that news items, their order, and a reading manuscript are given. The reading part is specified. The forced alignment is an algorithm for obtaining correspondence between each phoneme and a speech voice section when a phoneme string of speech content is given. In this system, it is possible to specify which part of the original is being read in units of phonemes, so that it is possible to determine a caption sentence at an early stage and output a caption with high consistency with a program video.

  Patent Document 1 discloses a technique for correcting a speech recognition error by comparing a recognized word string and a read original in units of words. The text correction device described in Patent Document 1 is characterized in that a word string in a speech recognition result is compared with a word string in a document in a word chain block of length N.

JP 2012-128188 A

JE Garcia, A. Ortega, E. Lleida, T. Lozano, E. Bernues, D. Sanchez, Aragon Inst. For Eng. Res., Univ. Of Zaragoza, Zaragoza, "Audio and text synchronization for TV news sub-titling based on Automatic Speech Recognition ", BMSB '09., IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, 2009

  The algorithm employed by the system described in Non-Patent Document 1 has a problem that the accuracy is significantly lowered when there is a discrepancy between the uttered speech and the phoneme string.

  On the other hand, in many news programs, it is difficult to specify news items and their order in advance. The order of news items is often changed during the broadcast of a program. In the example of CARTV described in Non-Patent Document 1, since a large-scale system linked with a news editing computer system (NRCS) is constructed, it is used for alignment whenever a news item or item order is updated. The phoneme sequence is updated.

  However, even if Non-Patent Document 1 adopts such a method, if the phoneme string update based on the management by NRCS is not in time, or if the document is skipped or paraphrased, appropriate alignment is performed. become unable. Therefore, in order to correct the alignment position, it is necessary to construct a complicated and large-scale system by acquiring and using a lap time that is a guide for program progression and the playback timing of the insert video. In addition, there are parts of the program where the content of utterances is not managed by the NRCS, such as an interview of an insert video. Therefore, in order to prevent the alignment from progressing in such a portion, complicated control such as managing the above-described insert video and stopping the input to the alignment based on the reproduction information is also required.

  For example, in a local broadcasting station that requires an automatic caption production system, it is difficult to construct and maintain such a complicated system, and a simpler system is required.

  Further, although the technique of Patent Document 1 does not require the complicated system configuration as described above, it does not estimate the sentence boundary of the recognition word hypothesis sequence, so that an inappropriate manuscript section and words of speech recognition result As a result of collating with the column, if there is a recognition error at the sentence boundary or block boundary, skipping or paraphrasing the manuscript, or adding an utterance that is not in the manuscript, the word is missing at the block boundary Or automatic correction errors such as the same word being output twice. Such an automatic correction error is the same even if the unit of the section in which the speech recognition result corresponds to the manuscript is set as a sentence or other unit as long as the boundary (sentence boundary) of the word hypothesis sequence based on the speech recognition result is unknown. Arise.

  In order to solve the above problems, a finite state transducer that allows recognition errors (that is, allows state transition even if an incorrect recognition result is input) is constructed in advance based on a broadcast document. It is also possible to correct the recognition error by calculating the likelihood using the state transition definition. However, in order to correctly estimate the current state using a finite state transducer that also allows recognition errors, it is necessary to satisfy a predetermined precondition. The precondition is that there are no similar sequences (for example, word sequences) in the document. If a similar sequence exists, the finite state transducer may be able to make a state transition even if an incorrect recognition result is input, so that the recognition error may not be corrected correctly. As a result, it is possible that the wrong manuscript is transmitted as subtitle text. When subtitle text is used for broadcasting, it becomes a factor of reducing the accuracy.

The present invention has been made in view of the above circumstances, and provides a speech recognition error correction apparatus that can correct speech recognition errors more accurately.
In particular, the present invention provides a speech recognition error correcting apparatus that can more accurately estimate a node in a finite state transducer when correcting a speech recognition error.

In order to solve the above problems, a speech recognition error correction apparatus according to an aspect of the present invention includes a document text storage unit that stores document text, and language elements included in the document text stored in the document text storage unit. When a distance representing the similarity between a plurality of series is calculated for a series, a similar series is detected based on whether the calculated distance is less than a predetermined threshold, and a similar series is detected Outputs warning information, outputs a suggestion of correction for eliminating the similarity by manipulating the manuscript text for at least one of the detected similar sequences, and according to instructions input from the outside A preprocessing unit for correcting the manuscript text and a word corresponding to the manuscript text while sequentially receiving words inputted as a speech recognition result Information regarding the state of the finite state transducer that makes a transition and that makes a state transition while accepting an error in a word included in the speech recognition result, and a transition source state with respect to the state transition Finite state transducer information storage unit for storing information including transition destination state, input symbol, output symbol, and transition weight, and node data for storing a score representing the likelihood of the state in the finite state transducer A storage unit and an input of a recognition word that is a speech recognition result corresponding to the original text are received from the outside, and a score at the time for each state in the finite state transducer according to the received recognition word And the node data storage unit using the calculated score When a node data update unit to be updated and an activation signal indicating the start of processing are received from the outside, the maximum likelihood node at that time is determined by referring to the node data storage unit, and the finite state transducer information storage unit By referring back to the node data storage unit, trace back processing up to a predetermined time when the state transition has been confirmed by tracing back the state transition to the maximum likelihood node, and the path of the state transition subjected to the trace back processing And an error score according to the ratio of the state transition related to the error to the path is calculated for the path that is the output candidate, and the degree of error is calculated based on the calculated error score. The document search unit that determines the output candidate of the path as a definite output when the value is smaller than the predetermined threshold, and the document search unit. And an output unit for outputting the determined output obtained as described above.
The language unit is specifically a word, a character, a phoneme, or the like.

  According to another aspect of the present invention, in the above speech recognition error correction apparatus, the document text includes a plurality of items, and the preprocessing unit has the distance between the items below a predetermined threshold. In this case, the warning information indicating that the items are duplicated is output.

  According to another aspect of the present invention, in the above speech recognition error correction apparatus, the document text includes a plurality of sentences, and the preprocessing unit is configured such that the distance between the sentences is a predetermined threshold value. When the value is less than, the warning information indicating that the sentence is similar is output, and any of the similar sentences is connected to the sentence before the sentence or the sentence after the sentence. A correction proposal is output.

  According to another aspect of the present invention, in the above speech recognition error correction apparatus, the document text includes a plurality of sentences, and the sentences include one or more chunks. The pre-processing unit outputs the warning information indicating that the head chunks are similar when the distance between the head chunks located at the head of the sentence is below a predetermined threshold, and the similar It is characterized by outputting a proposal for a modification that extends at least one of the head chunks backward.

  According to another aspect of the present invention, in the above speech recognition error correction apparatus, the document text includes a plurality of sentences, and the preprocessing unit has a predetermined number of words included in the sentence. When the value is smaller than the threshold value, the warning information indicating that the sentence is too short is output.

  According to the present invention, the preprocessing unit detects similar expressions (similar sequences) in the document and eliminates them in advance. Therefore, when a finite state transducer generated based on the document text processed by the preprocessing unit is used, the accuracy of state (node) estimation is improved when searching for a document. Therefore, accuracy when correcting speech recognition errors is increased.

It is a block diagram which shows typically the system containing the speech recognition error correction apparatus (system 1) on which the embodiment of this invention is a premise. It is a figure which shows typically the construction example of the weighted finite state transducer for a speech recognition error correction apparatus (method 1). It is a block diagram which shows typically the structure of a speech recognition error correction apparatus (method 1). It is FIG. (1) for demonstrating the trace back and original division by a speech recognition error correction apparatus (method 1). FIG. 5 is a diagram (part 2) for explaining traceback and document division by the speech recognition error correction apparatus (method 1); It is a flowchart which shows the flow of a process by the speech recognition error correction apparatus (method 1). It is a schematic diagram which shows the example of the algorithm applicable with a weighted finite state transducer regarding a speech recognition error correction apparatus (method 1). It is a block diagram which shows the outline of a structure of the system containing the speech recognition error correction apparatus (system 2) on the assumption of embodiment of this invention. It is a state transition diagram which shows the example of WFST (weighted finite state transducer) for a speech recognition error correction apparatus (method 2). It is a state transition diagram which shows the example of WFST provided with the characteristic for using in a speech recognition error correction apparatus (method 2). It is a functional block diagram which shows schematic function structure of a speech recognition error correction apparatus (method 2). It is a flowchart which shows the procedure of the process by the speech recognition error correction apparatus (method 2). It is the schematic which shows the example of the confirmation method of the subtitle sentence output by a speech recognition error correction apparatus (method 2). It is a block diagram which shows schematic function structure of the speech recognition error correction apparatus by 1st Embodiment of this invention. It is a block diagram which shows schematic function structure of the speech recognition error correction apparatus by 2nd Embodiment of this invention. It is the schematic which shows the 1st example of the duplication item detected by the pre-processing part in 1st Embodiment or 2nd Embodiment. It is the schematic which shows the 2nd example of the duplication item detected by the pre-processing part in 1st Embodiment or 2nd Embodiment. It is the schematic which shows the example of the sentence pair detected by the pre-processing part in 1st Embodiment or 2nd Embodiment, and the example which connects one of the sentences with another sentence. It is the schematic which shows the example of the similar chunk detected by the pre-processing part in 1st Embodiment or 2nd Embodiment, and the example which proposed extension of those chunks. It is a program pseudo code which shows the calculation procedure for the pre-processing part in 1st Embodiment or 2nd Embodiment to calculate the distance between series (Improved version of Levenshtein distance). It is the schematic which shows the example of execution when the pre-processing part in 1st Embodiment or 2nd Embodiment calculates distance by said program.

  In the following, a two-type speech recognition error correction apparatus premised on an embodiment of the present invention will be described first, and then a plurality of embodiments of the present invention will be described.

[Method 1: Assumed form of speech recognition error correction device]
Hereinafter, the speech recognition error correction apparatus of this embodiment will be described in detail.
The speech recognition error correction apparatus 100 shown in FIG. 1 accepts as input a word string (recognition word string) of a recognition result output by the speech recognition apparatus 220 that recognizes the uttered speech read out from the document 201 included in the document text set 200. The error contained in the recognized word string is corrected by estimating the word string of the corresponding document stored in advance. Here, the information stored in advance for the estimation process by the speech recognition error correction apparatus 100 is a set of corresponding manuscripts constructed by reading the manuscript text set 200 in advance, and the state transition between nodes representing the state Is a corresponding manuscript set represented by a weighted finite state transducer (hereinafter referred to as WFST) having a branch representing the network as a network. This speech recognition error correction apparatus 100 sequentially checks the best hypothesis on the WFST network and does not wait for the input of all words based on the edit distance between the word string of the corresponding manuscript on the WFST and the recognized word string. The best hypothesis is approximated and the correction results are partially determined sequentially.

  The example shown in FIG. 1 schematically shows the entire system including the speech recognition error correction apparatus 100 to be applied when subtitles are given to a news program originated from a local broadcasting station using speech recognition. The utterance voice of such a program is characterized in that it is generally based on a manuscript text prepared in advance. In addition, large-scale key broadcasting stations have operators to manually correct recognition errors included in speech recognition results, but local broadcasters currently have difficulty in locating the operators. An example of this is shown. According to the present embodiment, it is possible to solve the problem of local station operator arrangement.

A manuscript text set 200 shown in FIG. 1 represents the whole of a text that is a transcript of what a person plans to speak. The manuscript text set 200 is divided into a number of subdivided individual contents, for example, according to word string delimiter units such as sentences, sentences, paragraphs, and content classifications such as themes and topics. Such individual contents are hereinafter simply referred to as a manuscript. Further, description will be made assuming that the word string unit is a sentence as an example.
The document text set 200 is stored in the document text storage unit. The document text storage unit is configured using, for example, a magnetic disk device or a semiconductor memory.

In the present embodiment, for example, the following conditions (A1) to (A7) are assumed.
(A1) A plurality of manuscript sentences in the manuscript text set 200 are read as voice recognition targets.
(A2) For example, even if the manuscript is related to one news item, several updated versions (versions) are prepared, and it is not known in advance which version of the manuscript will be read in the news program. .
(A3) The order in which a plurality of manuscript sentences are read is not known in advance.
(A4) Some original texts included in the original text set 200 cannot be read.
(A5) Depending on the person reading, there is a case where the wording is intentionally changed or a wording error occurs without reading the document as it is.
(A6) A major premise is to avoid sending a subtitle that has become unclear due to a recognition error of the voice recognition device 220 to mislead or make the viewer uncomfortable. Therefore, it is not sent in the case of an unknown recognition result, and instead, the subtitle is a manuscript (prior manuscript) that is automatically estimated to be the closest to the utterance content that has been proofread by the editor and confirmed in advance. Send it out.
(A7) If there is no original corresponding to the recognition result in an interview part or the like, automatic estimation is impossible, so no subtitles are transmitted for an interview part without an original.

  The manuscript text set 200 is a set of manuscript electronic data submitted by a reporter for, for example, a news program, and is stored in a general storage device such as a hard disk or a storage means on a network. This document text set 200 is also used to construct a WFST of the corresponding document set in advance.

  When raw speech data is input, the speech recognition device 220 recognizes speech data using an acoustic model or language model based on a hidden Markov model (HMM), and generates the recognized result as a recognition word string. To do. In the present embodiment, the voice recognition device 220 is not particularly limited, and a conventionally known device can be adopted.

  As shown in the condition (A2), a plurality of versions of a manuscript have been submitted for each news item, and it is not possible to determine in advance which version will be broadcast in which order. In such a situation, the speech recognition apparatus 220 must perform speech recognition and immediately check whether or not a corresponding document exists for the uttered speech. Therefore, the language model used for speech recognition is adapted using the document text set 200 in order to obtain a high-accuracy correspondence between the speech recognition result and the document, and the recognition accuracy when it is read out as it is according to the document becomes high. It is preferable to do so.

The transducer construction device 240 constructs a WFST as a collection of corresponding originals (corresponding original collection) used in the speech recognition error correction apparatus 100.
The transducer construction device 240 constructs in advance a WFST to be used by the speech recognition error correction device 100 from the read-out original that is the target of speech recognition, that is, from the original text included in the original text set 200. WFST is a finite state machine having input symbols, output symbols, and transition weights, and can efficiently handle input / output of different granularities such as words and sentences. The construction of this WFST will be described later.

  Each time a recognition result word is input from the speech recognition device 220, the speech recognition error correction device 100 uses WFST to obtain a transition that can accept the input word, calculates its score, and sets a threshold for the cumulative score. It is assumed that a search (Viterbi search) using a conventionally known Viterbi algorithm is used while performing pruning. Note that the Viterbi algorithm is a method that uses a trellis diagram to effectively calculate a code sequence having the maximum likelihood when estimating a code sequence that is closest to a transmission code with respect to a received sequence, that is, that maximizes the likelihood. It is a method of searching.

  In normal Viterbi search, after all the inputs are observed, the path with the best score is traced back and the best hypothesis is output. Therefore, in the normal search method, it is not possible to output the correction results sequentially from the oldest input before all the inputs are observed. For example, assuming that the subtitles are produced from the result of recognizing the broadcast sound of a TV broadcast program and superimposed on the image of the TV screen in real time, the maximum likelihood sequence by the normal Viterbi search must input words until the end of the program It cannot be confirmed. This will end the program, so normal Viterbi search is not suitable for such operations.

  On the other hand, the speech recognition error correction apparatus 100 successively approximates the maximum likelihood sequence and traces back using the Viterbi search. That is, every time a predetermined processing start condition is satisfied, a path with the best score at that time is traced back and an output transition that can be determined is determined, so that correction results can be output sequentially. The path traced back here is an approximation of the best hypothesis, but whether or not the path is determined based on the reliability of the edit distance between the input word string corresponding to each output transition and the word string of the manuscript. To improve the approximation accuracy. Details will be described later.

[Example of constructed WFST]
FIG. 2 shows an example of WFST constructed by the transducer construction device 240. The WFST has a node representing a state and a branch representing a state transition. Note that state transition may be simply referred to as transition. In the present embodiment, a WFST is constructed in which an input symbol is a word and an output symbol is a predetermined word string. A predetermined word string will be described as a sentence.

  In this example, each ellipse node is given a three-digit number for identification. The start point node is node 001, and the end point node is node 008. In this example, nodes 002 to 007 are arranged in a straight line between the start point and the end point. Further, nodes 010 to 015 are arranged in a straight line in parallel between the start point and the end point. Furthermore, nodes 018 to 023 are arranged in a straight line in parallel between the start point and the end point. In this example, a transition (branch) is set between the state (node) and the state (node). Here, when it is between nodes, it also includes between own nodes. Each transition has a word or symbol <S>, <I>, <D>, <EmiX (where X is one of 1 to 3)> and <eps> Either is listed.

First, to generalize all the transitions in FIG. 2, in this WFST, a parameter of (S ⁱ / S ^o : ω) is set for each transition between states. Here, S ⁱ represents a word input accepted by the transition, S ^o represents a predetermined word string (sentence) output by the transition, and ω represents a transition weight (state transition weight). That is, a triple parameter is set for each transition. However, for the convenience of the paper in FIG. 2, not describe parameters in all transitions, the transition of the total 18 words have been described, S ⁱ of the three sets of parameters, one of the S ^o Only is described.

  Here, the word described in FIG. 2 is generalized and expressed as a word s. Note that uppercase and lowercase letters are distinguished. In FIG. 2, a word s represents a word included in the word string of the document. Each transition in which the word s is described indicates that transition is possible only when the same word as the word s described in the transition is input. That is, if the word input at the position of the recognized word string corresponding to the position of a certain word s included in the word string of the document is the same as the word s on the document, the state can be changed. In short, each transition in which the word s is described is a transition that proceeds after receiving a speech-recognized word. As described above, the WFST constructed here is a network that can freely connect all document texts.

  In FIG. 2, the parameter for the transition in which the word s is described is represented by (s / ε: 0.0). Here, s represents a word input that the transition can accept, and ε means that there is no output in this transition. Moreover, 0.0 is one of the transition weights, and means that no penalty is imposed when the same word as the word s is input for this transition. For example, a transition in which “last month” is described in FIG. 2 is (last month / ε: 0.0) in terms of a triple parameter.

  In FIG. 2, a transition in which <S> is described is a transition for accepting a replacement word. That is, when the word input at the position of the recognized word string corresponding to the position of a certain word s included in the document word string is replaced with an arbitrary word different from the word s on the document, the replacement is performed. A transition for accepting a word. Hereinafter, an arbitrary word different from the word s at the position of a certain word s included in the word string of the document is referred to as an arbitrary word *. This replacement includes, for example, a case where “restart” is recognized as a transliteration of the homonym “reunion”.

In FIG. 2, the transition in which <S> is described can accept any word *. The parameter for the transition in which <S> is described is represented by (* / ε: ω _s ). Here, * represents an arbitrary word input that the transition can accept, and ε means that there is no output in this transition. Further, ω _s is one of transition weights, and means a penalty imposed on the transition when an arbitrary word * different from the word s is input (hereinafter referred to as a replacement penalty). This replacement penalty ω _s is expressed by a numerical value that lowers the node score, and for example, −1.0 is used. For example, the transition in which <S> is described in FIG. 2 is (* / ε: −1.0) in terms of a triple parameter.

  In FIG. 2, a transition in which <I> is described is a transition for accepting an insertion word. In other words, when there is a repetition of information added to the utterance content or segmentation due to stagnation due to the speaker, the word inserted at the position following the word string recognized as being replaced or replaced is accepted. Transition. In addition, due to the voice recognition device 220, a word inserted in a position following the one word as the manuscript is caused by a recognition error that recognizes a word that should be recognized as the manuscript as a plurality of words if it is as the manuscript. Is a transition to accept

In FIG. 2, the transition with <I> described can accept any word *. The parameter for the transition in which <I> is described is represented by (* / ε: ω _i ). Here, * represents an arbitrary word input that the transition can accept, and ε means that there is no output in this transition. Further, ω _i is one of transition weights, and means a penalty imposed when an arbitrary word * is input for this transition (hereinafter referred to as an insertion penalty). This insertion penalty ω _i is represented by a numerical value that lowers the node score, and for example, −1.0 is used. For example, the transition in which <I> is described in FIG. 2 is (* / ε: −1.0) in terms of a triple parameter.

  In FIG. 2, a transition in which <D> is described is a transition for accepting a dropped word. That is, this is a transition for specifying the position of a word dropped from the original in the recognized word string when a phrase or the like is dropped in a part of the utterance content due to the speaker. In addition, due to the voice recognition device 220, a word that should be recognized as a plurality of words according to the original is caused by a recognition error in which the word is deleted and recognized as one word, and is dropped from the original in the recognized word string. It is a transition for specifying the position of a word.

In FIG. 2, transitions with <D> can be transitioned even if no word is input. The parameter for the transition in which <D> is described is represented by (ε / ε: ω _d ). Here, the first ε means that there is no word input in this transition, and the next ε means that there is no output in this transition. Further, ω _d is one of transition weights, and means a penalty imposed when a word is dropped in this transition (hereinafter referred to as a drop penalty). This drop penalty ω _d is represented by a numerical value that lowers the node score, and for example, −1.0 is used. For example, the transition in which <D> is described in FIG. 2 is (ε / ε: −1.0) in terms of a triple parameter.

  In FIG. 2, a transition in which <EmiX> is described is a transition for outputting a sentence L as a predetermined word string, and a transition for outputting a correction result. The parameter for the transition in which <EmiX> is described is represented by (ε / L: 0.0). Here, ε means that no word is input in this transition. L means a word string (sentence) output in this transition. For example, a transition in which <Emi1> is described in FIG. 2 is (ε / Last Kanto Koshin region ...: 0.0) in terms of a triple parameter. That is, in this case, L is a word string in which all of the word strings “Kanto Koshin region in last month are ...” arranged in each transition from the start node 001 to the node 007 via the node 002 are connected in order. The parameter 0.0 is one of transition weights, which means that no penalty is imposed when a sentence is output for this transition.

In FIG. 2, the transition in which <eps> is described is a transition that connects the end point node and the start point node, and is called an epsilon transition (ε transition). The transition in which <eps> is described is a transition that gives a constraint that a predetermined word string (sentence) included in the document text set is continuously spoken. A parameter for a transition in which <eps> is described is represented by (ε / ε: ω _u ). The first ε means that there is no word input at this transition, and the next ε means there is no output at this transition. Also, ω _u is one of transition weights, and by giving an appropriate weight (numerical value), the WFST can increase the score of sentences that match longer.

[How to build WFST]
A WFST construction method by the transducer construction device 240 will be described.
A word string unit for arranging output transitions (transitions in which <EmiX> is described) in WFST is determined in advance. This can be set according to the required error correction capability. The positions where the output transitions are arranged can make the unit of the manuscript included in the manuscript text set 200 as one delimiter. As the position where the output transition is arranged, a sentence unit, a phrase unit, a line feed unit arranged for easy reading by the reporter, or the like can be used. Here, when a long unit is set, the correction accuracy increases, but the determination of the subtitle word string to be transmitted is delayed. Conversely, if a short unit is set, the determination of the subtitle word string to be sent out becomes faster, but the correction accuracy decreases. Therefore, what unit should be used may be appropriately designed according to the expected recognition accuracy of voice recognition and the degree of matching between the original and the reading voice.

  In this embodiment, the position where the output transition is arranged in the WFST is determined in units of sentences as an example. From another viewpoint, the WFST in FIG. 2 uses a branch (word s in FIG. 2) that represents input transition of each word constituting a sentence between a start point node 001 and an end point node 008 for each sentence (a word string in a predetermined unit). ) And a branch representing an output transition (transition in which <EmiX> is described in FIG. 2).

  The construction of the WFST starts with the starting point of the WFST, and each time the original text included in the original text set 200 is read word by word, a transition with a weight of 0 for accepting the word and a new node are sequentially created. . Here, the transition of weight 0 is (s / ε: 0.0) when expressed by a triple parameter. And when it becomes the above-mentioned predetermined unit, an output transition is added and it connects with the end point node of WFST. If the original still remains, starting from the starting point again, each time the original text is read word by word, a transition with a weight of 0 and a new node for accepting the word are sequentially created. And when it becomes the above-mentioned predetermined unit, an output transition is added and it connects with the end point node of WFST. Thereafter, the same is repeated.

When all the original texts have been read from the original text set 200, finally, the end point node and the start point node are connected by the ε transition. Here, the ε transition is (ε / ε: ω _u ) in terms of a triple parameter. Here, an appropriate weight is given to the transition weight ω _u . As a result, the WFST can increase the score of a sentence that matches longer, and can operate properly even when a sentence that matches the prefix of another sentence exists in the document. Finally, transitions that accept substitutions, omissions, and insertions are added to the transitions of each word.

[Configuration example of transducer construction device]
In the example shown in FIG. 1, the transducer construction device 240 includes a word network registration unit 241 and an editing network registration unit 242.
The word network registration unit 241 performs the following series of processes for each predetermined unit (for example, sentence unit) in the document text included in the document text set 200. That is, as a series of processes, the word network registration unit 241 receives input words from the start node of the WFST network each time a word in a word string included in the original text included in the original text set 200 is read. Branches and new nodes are sequentially created until the read word string reaches a predetermined unit (for example, sentence unit). Then, in the WFST network, an output transition branch of the read word string is added and connected to the end node.

  The editing network registration unit 242 corresponds to a branch representing a state transition that accepts an arbitrary word corresponding to a word replacement and a word insertion between nodes of the WFST network created by the word network registration unit 241. Thus, a branch representing a state transition that accepts an arbitrary word and a branch representing a state transition that transitions to the output side even if there is no input in response to the deletion of the word are added.

[Configuration example of speech recognition error correction device]
In the example shown in FIG. 1, the transducer construction device 240 is provided separately from the speech recognition error correction device 100, but as shown in FIG. 3, for example, the speech recognition error correction device 100 includes the transducer construction device 240. May be. As shown in FIG. 3, the speech recognition error correction apparatus 100 includes a WFST storage unit (corresponding document set storage unit) 110, a node data update unit 120, a node data storage unit 130, a document search unit 140, and a document. Output means 150.

  The WFST storage unit (corresponding document set storage unit) 110 stores a WFST (corresponding document set) constructed in advance using the document text set 200. This WFST (corresponding document set) is constructed by the transducer construction device 240. Therefore, since WFST is the same as that described with reference to FIG. 2, description thereof is omitted to avoid duplication.

  The node data updating unit 120 calculates and updates a score of a state that can be transited on the WFST network as node data every time when an input of a word of a recognized word string output from the speech recognition device 220 is received. For example, each time a recognition word is input, the node data updating unit 120 refers to the WFST stored in the WFST storage unit 110 and sequentially performs a Viterbi search to update the node data.

The node data updating unit 120 adds “0” to the score when the word input as the recognition word string is the same word as the corresponding document, and penalizes the score when the input word is a word different from the corresponding document. Add "-1".
For example, in the example shown in FIG. 2, when the word string input as the recognition word string is exactly the same word string as the corresponding document, the word “Last Month” is received from the start node 001 and corresponds to the word of the corresponding document. Since the process proceeds to the node 002 through the transition, the node data update unit 120 adds “0” to the score. Thereafter, for example, when “NO” is received and the process proceeds to node 003, “0” is added to the score. Similarly, when “Kanto Koshin”,... Is received, “0” is added to the score.

  On the other hand, for example, in the example shown in FIG. 2, when the word string input as the recognition word string is a word string different from the corresponding manuscript, if the word “Last Week” is received from the start point node 001, ”Has been replaced, the process proceeds to node 002 through a transition corresponding to the replacement. In this case, the node data updating unit 120 adds a penalty “−1” to the score. Also, when passing through a transition corresponding to an insertion error or dropout error, the node data update unit 120 similarly adds a penalty “−1” to the score.

  Thus, when the input recognition word is the same as the word s in WFST, the score of the path is the best. On the other hand, if there is a replacement, insertion, or deletion edit, the score deteriorates. For example, a transition in which <D> is described can transition even if there is no input. However, in the case of a path that passes only a transition in which <D> is described, the score becomes lower as it approaches the output transition. The WFST is created as a network in which if the recognition word string includes an error or paraphrase, the score is deteriorated accordingly.

  The node data storage unit 130 stores the node data calculated by the node data update unit 120 at each update time, and is a general storage unit such as a memory or a hard disk.

  The document searching means 140 is stored at that time every time a predetermined processing start condition is satisfied without waiting for input of recognition results of all recognized word strings for all documents for determining the final best hypothesis. A hypothesis partially approximating the final best hypothesis is sequentially determined as an error correction result while tracing back on the WFST network based on the node data.

  The document search unit 140 approximates the final best hypothesis based on the edit distance between the word string of the corresponding document included in the WFST (corresponding document set) and the input recognition word string. The document search means 140 determines that the path section is reliable if the edit distance in the path section from the head to the end is small to some extent between the paths divided in predetermined ranges on the WFST network. And output. Here, the short edit distance means that the path through which the recognized word string and the original word string are almost matched has been passed. On the other hand, since the reliability of a path section with a long editing distance is low, it is not determined at that time and is used for the next traceback. It is presumed that a path section having a low reliability forever is a section talking about a difference that is not originally described in the manuscript. Therefore, a path section with low reliability is not output. In the following, a path section in a predetermined range on the WFST network will be described as an example of a path section sandwiched between two output transitions on the WFST network.

  The processing start condition is satisfied when, for example, a silent period without speech is reached a predetermined period, or the number of input words as a recognized word string output by the speech recognition device 220 is a predetermined number of words. It means the case of reaching. The predetermined period is not particularly limited, but an example is 3 seconds. Further, the predetermined number of words is not particularly limited, but 20 words can be given as an example. The activation signal at this time may be automatically output to the voice recognition device 220, for example, or manually when the operator recognizes that it is a pose or when a predetermined number of words has been reached. You may make it input. According to this, processing load can be reduced compared with the case where a search process is started for every input of a recognition word. Further, for example, if the silent period is only a predetermined period, since the sequential reception of the recognition results is stopped during that period, the node scores at that time can be easily compared.

  In order to realize the above function, in the present embodiment, the document search unit 140 includes a maximum score node detection unit 141, a traceback unit 142, a document division unit 143, an output candidate, as shown in FIG. The storage unit 144, the edit distance calculation unit 145, the edit distance determination unit 146, the fixed output storage unit 147, and the fixed time storage unit 148 are provided.

  The maximum score node detection unit 141 detects a node having the maximum score in the node data stored at that time when a predetermined process start condition is satisfied. For example, when the silent period (pause) in which there is no uttered voice reaches a predetermined period, or whenever the number of input words as a recognition result reaches a predetermined number of words, the activation signal indicating that is the maximum score Input to the node detection means 141.

  The traceback means 142 follows the WFST network from the downstream to the upstream for the path that has reached the node from the node detected by the maximum score node detection means 141, and is confirmed by the previous traceback, and the output word Trace back to the time corresponding to the last input word of the sequence.

  FIG. 4 is a schematic diagram in which a path P1 is added to the WFST shown in FIG. In FIG. 4, it is assumed that the node with the highest score is the node 020. Further, it is assumed that the node corresponding to the last input word determined in the previous traceback is the node 007. In this case, the traceback unit 142 traces the path P1 from the position indicated by the star in the reverse direction in the order of the node 020, the node 019, and the node 018, and when it reaches the start point node 001, returns to the end point node 008. Next, the node 015 is reached through the output transition <Emi2> of the second tree. Subsequently, the traceback unit 142 traces backward in the order of the nodes 015, 014,... In FIG. 5, and when reaching the start point node 001, returns to the end point node 008 as shown by the path P2. Next, the node 007 is reached through the output transition <Emi1> of the first tree.

Returning to FIG. 3, the description of the document search means 140 will be continued.
The document dividing means 143 cuts out a word string of the corresponding document included in the WFST (corresponding document set) for each path section sandwiched between two output transitions in the path traced back this time. In the case of the example described with reference to FIGS. 4 and 5, the path section sandwiched between the output transition <Emi1> and the output transition <Emi2> is divided by the document dividing unit 143.

  The output candidate storage unit 144 stores an output transition output symbol (cut out document) corresponding to the path section divided by the document dividing unit 143 as an output candidate. Storage means. In the case of the example described with reference to FIGS. 4 and 5, “This week's rain is ...” is stored as an output candidate.

The edit distance calculating unit 145 calculates an edit distance from the input recognition word string for each corresponding document cut out by the document dividing unit 143. In the present embodiment, the editing distance is defined by a value obtained by dividing the number of editing operations related to insertion, replacement, and deletion for the path section by the number of words in the path section. Here, assuming that the number of word replacement, insertion, and deletion editing operations in the recognized word string is e, and the number of original words corresponding to the output transition is N _r , the editing distance is recognized for the original word number N _r . It is represented by the ratio (e / N _r ) of the number of word editing operations e in the word string.

  Specifically, in the example shown in FIG. 2, the case where the path section on WFST is “node 007 → node 008 → node 001 → node 0010 → node 0011 → node 0012 → node 0013 → node 0014 → node 0015”. Suppose. This path section is composed of 6 words, and when the word “this week” is replaced with “this month” and recognized, the edit distance is 1/6.

The edit distance determining means 146 sequentially determines whether or not the calculated edit distance is equal to or less than a predetermined threshold while selecting a path section from the downstream to the upstream in the WFST network. The output transition of the path section on the WFST network is determined, and the output symbol is determined as an error correction result. Here, when the edit distance (e / N _r ) is equal to or smaller than the threshold T, that is, when e / N _r ≦ T is satisfied, the output symbol of the output transition is determined as the correction result. Further, when the edit distance (e / N _r ) is larger than the threshold value T, the edit distance determination unit 146 does not adopt the output symbol. That is, the output of the output transition of the path section having an edit distance larger than the threshold is temporarily suspended, and is rejected when there is an output transition determined after this path section. Since the edit distance (e / N _r ) is a value in the range of 0 to 1 from the definition, the threshold satisfies the relationship 0 <T <1.

  The definite output storage unit 147 stores the output symbol of the output transition in the predetermined path section as an error correction result when the editing distance determination unit 146 determines that the editing distance is equal to or less than the predetermined threshold. For example, it is a general storage means such as a memory or a hard disk. The storage structure of the definite output storage unit 147 is a stack and holds data in a last-in first-out structure.

  The fixed time storage unit 148 stores the fixed time determined by the current traceback process, and is a general storage unit such as a memory or a hard disk. The confirmation time storage means 148 is the latest confirmation corresponding to the output symbol stacked on the stack at the time when the edit distance determination means 146 has finished the determination for all the path sections (all cut out documents) to be traced back this time. The time of the word is stored as the confirmed time.

  The manuscript output unit 150 sequentially outputs the corresponding manuscript determined as an error correction result by the manuscript search unit 140. The manuscript output means 150 is determined by the end of the determination process for the edit distance calculated for each path section of all the cut out corresponding manuscripts in the path traced back through the WFST network this time. The data of the output symbols loaded on the stack is output until the stack becomes empty.

  The correction output by the speech recognition error correction apparatus 100 includes both the meaning of correcting an error and not outputting the error. In other words, if the correction result by the speech recognition error correction device 100 can be viewed in advance by a person, an error is enough to feel that “this is not a sentence” or “the meaning is different”. The operation in which the speech recognition error correction apparatus 100 detects the part in the process and does not output the detected part is included as error correction in a broad sense.

[Operation of voice recognition error correction device]
The flow of processing by the speech recognition error correction apparatus 100 according to this embodiment will be described with reference to FIG. 6 (see FIG. 3 as appropriate).
(Assuming 1) the recognition result of a word input _{{ω 0, ω 1, ...} , ω k, ..., ω j, ...} and.
(Assumption 2) _Let ω _k be the last input word of the part determined by the previous traceback, and let the output transition at that time be a _p (Pth output transition along the time axis).
(Premise 3) Consider a case in which, after a word ω _{j as a} recognition result is input, successive determination is performed using a predetermined silence as a trigger.
(Premise 4) The node data updating unit 120 accepts the last input word ω _j before silence and calculates all nodes that can make a transition.

The maximum score node detecting means 141 detects the node having the highest score in the node data stored at the present time, triggered by the predetermined silence being continued (step S1). The state represented by this detection node is the maximum likelihood state at the start of traceback. Then, the traceback unit 142 traces the word history on the WFST in the reverse direction for the path reaching the node from the detected node, confirms it in the previous traceback, and outputs the last input word of the output word sequence. Trace back to a fixed time corresponding to ω _k (transition in which the word accepted by the WFST transition is ω _k ) (step S2). Here, the confirmed time stored in the confirmed time storage means 148 is used as the confirmed time corresponding to the last input word ω _k of the word sequence that is confirmed and output in the previous traceback. Instead of the transition whose word is ω _k , traceback may be performed until the output transition a _P is reached.

Then, the document dividing unit 143 divides the document for each path section sandwiched between two output transitions in the path traced back this time, and stores the document in the output candidate storage unit 144 as an output candidate (step S3). . Here, the document is divided every time it passes through the output transition a _L (L-th (but L> P) output transition along the time axis) that can be output while proceeding in the reverse direction until reaching the output transition a _P. Alternatively, the document may be divided every time it passes through the output transition a _L that can be output from the output transition a _P side. The output transition a _L that can be output includes output transition symbols that are candidates for output, but also include output transitions that are later rejected by the edit distance determination unit 146 and are not output. The edit distance of such an output candidate is denoted as D.

Then, the edit distance calculation unit 145 calculates the edit distance D of the output candidate (step S4). Specifically, the section that corresponds to the output symbol of the output transition a _L, i.e., immediately before the output transition a _L-1 and with the output transition a _L of when it proceeds in the opposite direction on WFST from the output transition a _L A value obtained by dividing the number of editing operations (that is, the number of passes through <S>, <D>, and <I>) for the path section sandwiched between by the number of words in the section is the output transition a _L Is calculated as the edit distance D _L at. That is, assuming that the number of editing operations in the same section is e _L and the number of words in the same section is N _L ^r , the editing distance D _L in the output transition a _L is expressed by e _L / N _L ^r .

  Then, the edit distance determination unit 146 selects a predetermined output candidate, and determines whether or not the calculated edit distance D is equal to or less than the threshold value T (step S5). When the edit distance D is less than or equal to the threshold T (step S5: Yes), the edit distance determination unit 146 determines the output transition of the path section on the WFST and determines the output symbol as an error correction result (step S6). ). Further, the editing distance determination unit 146 accumulates the data of the output symbol determined this time on the stack stored in the finalized output storage unit 147 (step S7), and proceeds to step S8.

  Then, when there is an output candidate that should still be selected ahead (Step S8: No), the edit distance determination unit 146 returns to Step S5. On the other hand, when all the output candidate selections are selected (step S8: Yes), that is, when the determination process for the edit distance calculated for each path section corresponding to all the cut out documents is completed, The document output means 150 sequentially outputs the output symbol data stacked on the stack at that time until the stack becomes empty (step S9). As a result, the documents are output sequentially from the document placed on the front side.

  Here, when all the output candidate selections are selected (step S8: Yes), the edit distance determination unit 146 selects the latest confirmed word corresponding to the output symbol stacked on the stack for the current traceback process. Is stored in the fixed time storage means 148 as the fixed time determined in step.

  If the edit distance D is greater than the threshold T in step S5 (step S5: No), the process proceeds to step S8 without loading data on the stack.

  That is, the document output unit 150 sequentially outputs the data accumulated on the stack in each traceback process as a confirmed document. At this time, if the edit distance D of the predetermined path section is larger than the threshold T in the speech recognition result, the output symbol of the output transition of the path section is adopted as the error correction result because the path has low reliability. And no output.

[How to determine the edit distance threshold T]
If the recognition accuracy of voice recognition is about 90%, the edit distance value may be about 90%. The threshold value T of the edit distance used for the determination is preferably set at a position sufficiently lower than the recognition accuracy of voice recognition, for example, with a margin below the reliability of the word matching rate. Here, the reliability of the word match rate depends on the number of words between two output transitions of the WFST network.

As another factor, it is preferable to determine the threshold value T in consideration of the ratio of the overlapping of the original document candidates included in the original text set 200 as a ratio. For example, in the case of sentences shown in (E1) to (E3) below, overlapping as sentences is included at a rate of about 80%.
(E1) Today's weather is sunny (E2) Today's weather is rainy (E3) Today's weather is cloudy In this case, if the edit distance threshold is set to about 80%, the desired movement Cannot be realized. It has been confirmed that when an experiment is performed under the condition that an output transition is arranged for each sentence of the news manuscript and the threshold T is 50%, the operation is performed without any problem.

[WFST options]
<Option 1: Construction of WFST that accepts paraphrasing>
A manuscript serving as a WFST information source may include a phrase that is skipped when it is read, a phrase that can be rephrased, and a phrase that is supplemented. Some of these are typical and occur frequently. For example, in a news program manuscript, phrases such as “according to the Metropolitan Police Department” representing the source of the interview are fixed phrases that are easily skipped. However, even if this is skipped, there is no change in the meaning of the news main sentence (5W1H), and there is no practical problem.

  In option 1, such a standard wording is added to the WFST so that the correction result can be output with high accuracy. As conventionally known, WFST is used for speech recognition decoders, machine translation, and the like, and various calculation algorithms are known. For example, an algorithm that performs synthesis (see FIG. 7A), minimization (see FIG. 7B), and determinization (see FIG. 7C) can be applied, and an efficient state transition machine can be configured. There is a feature that can be done. The addition of the wording can be efficiently realized by separately constructing a WFST for adding wording separately from the WFST constructed from the manuscript and combining it with the WFST constructed from the manuscript.

  For example, as for paraphrasing examples, prepare the ones that have a high frequency and that have no change in meaning by paraphrasing, based on the difference between the original of the same kind of program in the past and the word string actually read out. deep. For each selected paraphrase example, a WFST for synthesizing paraphrases is constructed, and a WFST that is compatible with paraphrasing can be constructed by performing a synthesis operation with the WFST constructed from the original. Here, the synthesis of WFST will be described with reference to FIG.

  In FIG. 7A, the nodes are shown as circles. The upper diagram on the left side of FIG. 7A is a schematic diagram of an example of a WFST constructed from a document, and the lower diagram on the left side of FIG. 7A is a schematic diagram of an example of an added WFST. . The diagram on the right side of FIG. 7A is a schematic diagram of the WFST after the WFST constructed from the original and the added WFST are combined.

<Option 2: Option A when creating WFST>
When creating the WFST, the WFST may be minimized if necessary. Here, the minimization of WFST will be described with reference to FIG. The diagram on the left side of FIG. 7B is a schematic diagram showing an example of a WFST constructed from a document by a normal method. Here, a1 to a6 indicate different words.

The diagram on the right side of FIG. 7B is a schematic diagram of the WFST after minimizing the WFST constructed from the original by a normal method. In the WFST after minimization, nodes (states) are aggregated in consideration of the arrangement order (word position) for the prefixes (words a1, a2) common to the three word strings of the original WFST, and the minimum number Branches (transitions) are arranged.
According to WFST minimization, word strings (sentences) having the same prefix can be shared by the same transition, so that the amount of calculation can be reduced.

<Option 3: Option B when creating a WFST>
Further, when creating a WFST, if necessary, the WFST may be determinized. Here, determinization of WFST will be described with reference to FIG. The diagram on the left side of FIG. 7C is a schematic diagram of a WFST having the same shape as the WFST shown on the right side of FIG. However, in the transition in which the word a4 is described in FIG. 7B, the output sentence o1 is described instead. Similarly, in the transition in which the word a5 is described, the output sentence o2 is described instead, and the output sentence o3 is described instead of the word a6.

The diagram on the right side of FIG. 7C is a schematic diagram of the WFST after determinating the original WFST. The WFST after determinization is different from the original WFST in that the output sentence o3 is described in the previous transition (one left side).
The original WFST has a branch at the time of state transition from the second node from the left to the next node, and when the transition from the second node to the lower node in the figure, the output statement is o1 or o2. It turns out that it becomes decisive to become o3. Therefore, in order to output the estimation result as soon as possible, the position of the output sentence is changed in the WFST after determinization.

  According to WFST determinization, there is an advantage that the output sentence is moved to a transition with a unique prefix, so that the output sentence can be finalized. However, if the WFST is determinized when creating the WFST, it is necessary to change the setting so that the search process of the maximum likelihood hypothesis by the manuscript search means 140 can also cope. That is, it is necessary to shift the path section for calculating the edit distance before and after the output transition as compared with the case where WFST is not determinized. In addition, the threshold value T needs to be set to a stricter value (small value) so that the expansion and contraction of the preceding and following path sections can be absorbed.

[Other options]
This embodiment can also be applied to the generation of multilingual subtitles. For example, in the output transition in which <Emi2> next to node 015 of WFST shown in FIG. 2 is described, an English sentence corresponding to a Japanese sentence from nodes 010 to 015 is used as an output symbol, thereby supporting Japanese voice input. English subtitles can be generated. If you need to generate Japanese and English subtitles at the same time, you can use the English translation along with “This rain is all together this week”.

  As described above, the speech recognition error correction apparatus 100 according to the present embodiment recognizes a sentence without fixing a sentence boundary under the constraint that sentences in a document are continuously uttered in an arbitrary order. By taking the correspondence between the result and the manuscript in word units, the automatic correction error of the conventional block matching method is eliminated. On the other hand, in order to obtain a corrected output with higher accuracy, it is desirable that the output has a sentence or a unit equivalent thereto. In order to eliminate this contradiction and achieve both, the speech recognition error correction apparatus 100 uses a weighted finite state transducer (WFST) to determine the correspondence between the recognition result and the document.

  Then, the speech recognition error correction apparatus 100 compares the word string of the recognized word with the word string of the original document to determine where the word string matches the longest of the conventional block matching method (the technique of Patent Document 1). Collating in a range longer than a word chain block of length N (number of words N). Compared to the conventional block matching method, the section for matching the word string of the recognized word and the word string of the manuscript is not only the section corresponding to the word chain block, but the manuscript sentence, which is longer Match the entire sentence. Therefore, it is clearly known where to make the match, and automatic correction errors can be reduced more than before.

  As described above, the speech recognition error correction apparatus according to the present embodiment has been described based on the embodiment, but the present embodiment is not limited thereto. For example, it was decided by using the edit distance whether or not the reliability of the estimated correspondence manuscript with respect to the recognition word of the utterance speech is high. The insertion rate may be used or they may be used in combination.

  For example, in the output transition in which <EmiX> of WFST shown in FIG. 2 is described, “,”, “.”, Symbols, and the like that cannot be obtained as a result of speech recognition can be embedded in the output symbols according to the notation of the manuscript. . In this case, subtitles that are easier to read can be generated.

  In this embodiment, it is not essential to add subtitles. Also, if the content of the speech to be spoken is determined to some extent and the content can be obtained in advance, it is not necessarily premised on the sound of the broadcast program.

The configuration of the method 1 described above is organized as follows.
[1-1] A recognition word string output by a speech recognition device that recognizes speech uttered by reading out a manuscript included in a manuscript text set is received as an input, and a word string of the corresponding manuscript is estimated from a corresponding manuscript set stored in advance. A speech recognition error correction device for correcting an error included in the recognition word string, the corresponding document set constructed by reading the document text set in advance, and a node between the nodes representing the state Corresponding manuscript set storage means for storing the corresponding manuscript set represented by a weighted finite state transducer having a branch representing a state transition as a network, and for each time the word input of the recognized word string is received, the weight Node data that calculates and updates the score of a state that can be transited on a network of finite state transducers with a node as node data New means, node data storage means for storing the calculated node data at each update time, and predetermined without waiting for input of recognition results of all recognized word strings for all manuscripts for determining the final best hypothesis Each time a specified processing start condition is satisfied, a hypothesis that partially approximates the final best hypothesis is sequentially obtained as an error correction result while tracing back on the network based on the node data stored at that time. A speech recognition error correction apparatus, comprising: a document search unit to be determined; and a document output unit to sequentially output a corresponding document determined as the error correction result.

  [1-2] A weighted finite state transducer preliminarily constructed as the corresponding document set stored in the corresponding document set storage unit is used as the network as a start point node and an end point node for each corresponding document included in the corresponding document set. And a branch representing an input transition of each word constituting the word string of the corresponding manuscript and a branch representing an output transition of the word string, and represents a state transition that transitions from the end node to the start node A branch that represents a state transition that accepts an arbitrary word in response to a word replacement, a branch that represents a state transition that accepts an arbitrary word in response to an insertion of a word, and a deletion of a word Correspondingly, at least one of branches representing a state transition that makes a transition to the output side even when there is no input is provided.

  [1-3] The document search means determines in advance on the network of weighted finite state transducers as an edit distance between a word string of a corresponding document included in the corresponding document set and the input recognition word string. A value obtained by dividing the number of editing operations related to insertion, replacement, and deletion of the word sequence of the corresponding manuscript in the range of the path section by the number of words in the path section is calculated, and the edit calculated for each path section is calculated. The speech recognition error correction apparatus according to [1-2], wherein the final best hypothesis is approximated by comparing a distance with a predetermined threshold.

  [1-4] The document search means sequentially starts before the path section of the corresponding document in the weighted finite state transducer network at the time when the word string of the corresponding document whose edit distance is equal to or less than the threshold is determined. By going back to the corresponding manuscript whose output has been confirmed, the manuscript output means sequentially outputs the manuscripts of all the path sections whose edit distance is equal to or less than the threshold value from the upstream of the network, and all the edit distances greater than the threshold value are output. The speech recognition error correcting apparatus according to [1-3], wherein the corresponding document in the pass section is not output.

  [5] A weighted finite state transducer previously constructed as the corresponding manuscript set stored in the corresponding manuscript set storage means has a similar meaning as the word string included in the manuscript text set as the network. A branch that accepts a word string as a paraphrase candidate, and / or a word string that is included in the original text set and that may be dropped in a recognized word string that is output by the speech recognition apparatus. The speech recognition error correction apparatus according to [1-1] to [1-4], further including a branch that accepts a word string.

  [6] The manuscript search means is configured such that when the silent period without the uttered voice reaches a predetermined period, or the number of input words as a recognized word string output by the voice recognition device reaches a predetermined number of words. In this case, the speech recognition error described in [1-1] to [1-5] is traced back on the network of the weighted finite state transducer on the assumption that the processing start condition is satisfied. Correction device.

[Method 2: Assumed form of speech recognition error correction apparatus]
This embodiment will be described with reference to the drawings. In this embodiment, a technique is used to determine early the correspondence between a word string that is inevitably mixed with an error such as a speech recognition result and a manuscript that is an information source of the word string. Thereby, the error included in the speech recognition result is automatically corrected.

[1. System configuration to which this embodiment is applied]
FIG. 8 is a block diagram showing an outline of a system configuration including the speech recognition error correcting apparatus according to the present embodiment. As shown in the figure, the system includes a speech recognition error correction device 100, a manuscript text storage device 700, a speech recognition device 720, and a transducer construction device 740.

  The system shown in the figure includes a speech recognition error correction apparatus 600 for application when subtitles are added to a news program at a broadcasting station using speech recognition. The utterance voice in such a program is characterized in that it is generally based on a manuscript text prepared in advance. By the way, in a large-scale key broadcasting station, there is a case where an operator for manually correcting a recognition error included in a speech recognition result is arranged in order to broadcast subtitles. On the other hand, local broadcasters often find it difficult to locate such operators due to various circumstances. According to this embodiment, even in such a local broadcasting station, it becomes possible to correct the speech recognition apologization without arranging an operator.

The speech recognition error correction apparatus 600 accepts a recognition result word string (recognition word string) output from the speech recognition apparatus 720 as an input, and estimates a word string of a corresponding document stored in advance, thereby being included in the recognition word string. It corrects errors.
Here, the information stored in advance for the estimation process by the speech recognition error correction apparatus 600 is a set of corresponding originals constructed by reading in advance the original text stored in the original text storage device 700. This set of corresponding manuscripts includes a weighted finite state transducer (hereinafter referred to as “WFST”) having nodes representing states and branches (arcs) representing state transitions between the nodes as a network (directed graph). Or simply "finite state transducer").
The speech recognition error correction apparatus 600 sequentially checks the best (maximum likelihood) hypothesis on the WFST network, and based on the edit distance between the word string of the corresponding manuscript on the WFST and the recognized word string, the word strings of all recognition results. Without waiting for input, the final best hypothesis is approximated and the correction results are partially determined sequentially.

  In response to the recognition result word being input from the speech recognition device 720, the speech recognition error correction device 600 uses WFST to obtain a transition that can accept the input word, calculates the score, and accumulates the score. A search (Viterbi search) using a Viterbi algorithm is used while pruning using a threshold for. The Viterbi algorithm is an existing technique, and is effective in performing a search using a trellis diagram when estimating a code sequence closest to a transmission code with respect to a received sequence, that is, a code sequence having the maximum likelihood. It is a simple method.

  In normal Viterbi search, after all the inputs are observed, the path with the best score is traced back and the best hypothesis is output. Therefore, in the normal search method, it is not possible to output the correction results sequentially from the oldest input before all the inputs are observed. For example, assuming that the subtitles are produced from the result of recognizing the broadcast sound of a TV broadcast program and superimposed on the image of the TV screen in real time, the maximum likelihood sequence by the normal Viterbi search must input words until the end of the program It cannot be confirmed. This will end the program, so normal Viterbi search is not suitable for such operations.

  On the other hand, the speech recognition error correction apparatus 600 according to the present embodiment performs traceback by approximating the maximum likelihood sequence in a timely manner while using the Viterbi search. That is, every time a predetermined processing start condition is satisfied, a path with the best score at that time is traced back and an output transition that can be determined is determined, so that correction results can be output sequentially. The path traced back here is an approximation of the best hypothesis, but whether or not the path is determined based on the reliability of the edit distance between the input word string corresponding to each output transition and the word string of the manuscript. To improve the approximation accuracy. Details thereof will be described later.

The manuscript text storage device 700 stores manuscript text data. The manuscript text includes a plurality of articles. Each article includes a plurality of sentences.
The manuscript text storage device 700 stores the whole of what is scheduled to be spoken by a person in a text. The manuscript text storage device 700 stores texts representing a number of subdivided individual contents according to word string delimiter units such as sentences, sentences, and paragraphs, and content classifications such as themes and topics. Such individual contents are hereinafter simply referred to as a manuscript. The unit of the word string is a sentence as an example.
The document text storage device 700 is constructed by using a general storage means such as a hard disk device or a semiconductor memory. The document text storage device 700 may be accessed as a so-called cloud service through a communication network or the like.

The speech recognition device 720 recognizes speech uttered by a person (such as an announcer) reading the document 701 included in the document text storage device 700, and outputs a word string as a recognition result.
When raw speech data is input, the speech recognition device 720 recognizes speech data using an acoustic model based on a hidden Markov model (HMM) and a language model, and recognizes the recognized result as a recognition word string. Is generated as In this embodiment, the voice recognition device 720 is not particularly limited, and a device using a known technique can be employed.

  As will be described later, in a broadcast news program, a plurality of versions of a manuscript are submitted for each news item, and it is not possible to determine in advance which version will be broadcast in which order. In such a situation, the voice recognition device 720 is required to perform voice recognition and immediately check whether or not a corresponding document exists for the uttered voice. For this reason, the language model used for speech recognition is preliminarily adapted using text data stored in the document text storage device 700 so that the speech recognition result and the document can be accurately associated with each other. It is preferable that the recognition accuracy when reading is increased.

  The transducer constructing device 740 constructs a WFST as a set of corresponding manuscripts (corresponding manuscript set) used in the speech recognition error correcting device 600. The transducer construction device 740 constructs in advance a WFST to be used by the speech recognition error correction device 600 from a read-out document that is a target of speech recognition, that is, a document text included in the document text storage device 700. WFST is a finite state machine having input symbols, output symbols, and transition weights, and can efficiently handle input / output of different granularities such as words and sentences. The construction of this WFST will be described later.

  The transducer construction device 740 includes a word network registration unit 741 and an editing network registration unit 742 as shown in the figure.

The word network registration unit 741 reads the original text included in the original text storage device 700 and performs a series of processes described below for each predetermined unit (for example, sentence unit). That is, the word network registration unit 741 starts from the start point node of the WFST network, and every time a word included in the document text is read from the document text storage device 700, a state transition (input transition) for accepting the word is received. Create a branch and a new node. The word network registration unit 741 sequentially creates the above branches and nodes for each read word until the end of a predetermined unit (for example, sentence unit) in the document.
Then, in the WFST network, an output transition branch of the read word string is added and connected to the end node.

  The editing network registration unit 742 corresponds to a branch representing a state transition that accepts an arbitrary word corresponding to a word replacement and insertion of a word between nodes of the WFST network created by the word network registration unit 741. Thus, a branch representing a state transition that accepts an arbitrary word and a branch representing a state transition that transitions to the output side even if there is no input in response to the deletion of the word are added.

In this embodiment, the system operates under the following conditions (A1) to (A7), for example.
(A1) A plurality of document sentences among the document sentences stored in the document text storage device 700 are read out as speech recognition targets.
(A2) A number of updated versions (versions) are also prepared for a news manuscript for one news item, and it is impossible to determine in advance which version of the manuscript will be read in the news program.
(A3) The order in which a plurality of manuscript sentences are read is not known in advance.
(A4) Some original texts included in the original text storage device 700 are skipped without being read out.
(A5) Some readers (announcers, casters, reporters, etc.) do not read the original manuscript faithfully, but may change the wording in consideration of ease of transmission to the viewer, An error may occur.
(A6) We want to make it a priority to avoid sending unclear captions due to recognition errors of the speech recognition device 720 to mislead or uncomfortable the viewer. Therefore, it is not sent in the case of an unknown recognition result, and instead, the subtitle is a manuscript (prior manuscript) that is automatically estimated to be the closest to the utterance content that has been proofread by the editor and confirmed in advance. Send it out.
(A7) If there is no original corresponding to the recognition result in an interview part or the like, automatic estimation is impossible, so no subtitles are transmitted for an interview part without an original.

[2. General WFST example]
FIG. 9 is a schematic diagram illustrating an example of a general WFST. WFST is expressed as a definition of state transition. The state transition has a node representing the state and a branch representing the state transition. Note that state transition may be simply referred to as transition. In the illustrated WFST, an input symbol is a word and an output symbol is a predetermined word string. The predetermined word string is a sentence. In the figure, the nodes are indicated by ellipses. Further, the branches are represented by lines with arrows. That is, the branch has a direction.

  In the figure, each ellipse node is given a three-digit number for identifying the node. The start point node is node 001, and the end point node is node 008. In this example, nodes 002 to 007 are arranged in a straight line between the start point and the end point. Further, between the start point and the end point, the nodes 010 to 015 are arranged in a straight line in parallel with the above-described series of nodes 002 to 007. Furthermore, between the start point and the end point, the above-described series of nodes 002 to 007 and the series of nodes 010 to 015 are arranged in a straight line with nodes 018 to 023. Further, a branch representing a state transition is given a word or a symbol such as <S>, <I>, <D>, <Emi1>, <Emi2>, <Emi3>, <eps>. Further, the branch representing the state transition extends from the transition source node to the transition destination node.

The state transition shown in the figure is generalized and described. In this WFST, a parameter (S ⁱ / S ^o : ω) is set for each transition. Here, S ⁱ represents a word input accepted by the state transition. S ^o represents a word string output by the state transition. The output word string is a sentence or a part of a sentence. Ω represents a state transition weight. That is, a triple parameter is set for each transition. In FIG 9, for convenience, not illustrate all the parameters of a state transition has, in correspondence with each state transition, shows only one of the S ⁱ and S ^o of the three sets of parameters ing.

  Here, the word described in the figure is generalized and expressed as a word s. Note that uppercase and lowercase letters are distinguished. In the figure, a word s is a word included in the word string of the document. When a word s is added to a certain state transition, it indicates that the state transition occurs only when the word s is input. That is, in a certain state, when the word s is added to the state transition having the state as a transition source, the state transition occurs only when the word s is input as a word in the recognized word string. That is, the state transition in which the word s is described is a transition that proceeds by accepting the speech-recognized word s. When a state transition occurs, the transition destination state becomes the next state.

  In the state transition shown in the figure, the sequence of the nodes 002 to 007 corresponds to the word string “Last month /// Kanto Koshin / local / ha /. Here, the slash “/” is a word break. The series of nodes 010 to 015 corresponds to the word string “This week / Moon / Matched / Rain / Ha /. The series of nodes 018 to 023 corresponds to the word string “Meteorological Agency / Nii / Yori / Masu / To / ...”. As described above, the WFST constructed here is a network that can freely connect all document texts.

  In the figure, the parameter for the transition in which the word s is described is represented by (s / ε: 0.0). That is, this triplet parameter accepts the word s as input and represents no output. ε is a symbol indicating that there is no word. That is, the output other word strings are empty. The triple parameter represents that the transition weight value is 0.0. There are various methods for setting the weight value. For example, when there is no penalty, 0.0 can be used as the transition weight value, and a negative weight can be used as the penalty. For example, when the current state is estimated based on the definition of the state transition, the total of the transition weight values in a predetermined series can be used as a score. As an example, the state transition to which the word “last month” is attached in the same figure is represented by (last month / ε: 0.0) as a triple parameter.

  In the figure, a state transition with <S> is a transition for accepting a replacement word. That is, when the word input at the position of the recognized word string corresponding to the position of a certain word s included in the document word string is replaced with an arbitrary word different from the word s on the document, the replacement is performed. A transition for accepting a word. Hereinafter, an arbitrary word different from the word s at the position of a certain word s included in the word string of the document is referred to as an arbitrary word *. The asterisk is a notation representing a wild card. This replacement includes, for example, a case where “restart” is recognized as a transliteration of the homonym “reunion”.

In the figure, a state transition with <S> can accept any word *. The parameter for the transition in which <S> is described is represented by (* / ε: ω _s ). In this triplet representation, * represents any word input that the transition can accept, and ε means there is no output at this transition. Further, ω _s is one of transition weights, and means a penalty imposed on the transition when an arbitrary word * different from the word s is input (hereinafter referred to as a replacement penalty). This replacement penalty ω _s is expressed by a numerical value that lowers the node score, and for example, −1.0 is used. For example, a state transition with <S> can be expressed as (* / ε: -1.0) as a triple parameter.

  In the figure, a state transition with <I> is a transition for accepting an insertion word. In other words, if there is a repetition of information addition or segmentation due to utterance due to the speaker, etc., the word inserted at the position following the word string recognized as the original or replaced is accepted. It is a transition to do. In addition, due to the voice recognition device 220, a word inserted in a position following the one word as the manuscript is caused by a recognition error that recognizes a word that should be recognized as the manuscript as a plurality of words if it is as the manuscript. Can also be used for state transitions marked with <I>.

In the figure, the state transition with <I> can accept any word *. The parameter for the transition in which <I> is described is expressed by a triplet (* / ε: ω _i ). Here, * represents an arbitrary word input, and ε represents an empty word. That is, this state transition accepts an arbitrary word and has no output word. Further, ω _i is one of transition weights, and means a penalty imposed when an arbitrary word * is input for this transition (hereinafter referred to as an insertion penalty). This insertion penalty ω _i is represented by a numerical value that lowers the node score, and for example, −1.0 is used. For example, a state transition marked with <I> is (* / ε: −1.0) in terms of a triple parameter.
It should be noted that the transition destination state in the state transition with <I> is the same as the transition source state in the transition.

  In the figure, a state transition with <D> is a transition for accepting a dropped word. That is, this is a transition for specifying the position of a word dropped from the original in the recognized word string when a phrase or the like is dropped in a part of the utterance content due to the speaker. In addition, due to the voice recognition device 220, a word that should be recognized as a plurality of words according to the original is caused by a recognition error in which the word is deleted and recognized as one word, and is dropped from the original in the recognized word string. It is a transition for specifying the position of a word.

In the figure, a state transition with <D> is a transition that can occur even if no word is input. The triple parameter for the transition in which <D> is described is represented by (ε / ε: ω _d ). Here, ε before the slash indicates that there is no input accepted in this transition. Also, ε after the slash means that there is no output at this transition. Further, ω _d is one of transition weights, and means a penalty imposed when a word is dropped in this transition (hereinafter referred to as a drop penalty). This dropout penalty ω _d is represented by a numerical value that lowers the node score, and for example, −1.0 is used. For example, the transition in which <D> is described in FIG. 9 is (ε / ε: −1.0) in terms of a triple parameter.
It should be noted that the transition source and transition destination states in the state transitions marked with <D> are the same as the state transitions relating to the dropped words. For example, the state transition with <D> from the node 002 to the node 003 enables the transition from the node 002 to the node 003 even when the parallel word “no” is omitted (however, the above-described dropping) With a penalty).

  In the figure, the state transitions to which <Emi1>, <Emi2>, and <Emi3> are attached are transitions for outputting a sentence as a predetermined word string. The sentence output here is the correction result of the word string recognized by the speech recognition process. The parameter for the transition to which <Emi1>, <Emi2>, and <Emi3> are attached is represented by (ε / L: 0.0). Here, ε represents that no word is input in this transition. L indicates that the word string (sentence) output in this transition is L.

  For example, a state transition to which <Emi1> is attached is represented by three parameters (ε / Last Kanto Koshin region ...: 0.0). In other words, in this case, L is in order (in series) all the word strings “last month /// Kanto Koshin / local / ha / ...” arranged in each transition from the start node 001 to the node 007 via the node 002. N) Connected word strings. Note that 0.0 in the parameter is one of transition weights, and means that no penalty is imposed when a sentence is output for this transition. Similarly, the parameter of the state transition to which <Emi2> is attached is (ε / rains gathered in this week ...: 0.0). In addition, the parameter of the state transition to which <Emi3> is attached is similarly (ε / 0.0 by the JMA).

In the figure, a transition in which <eps> is described is a transition connecting an end point node and a start point node, and is called an epsilon transition (ε transition). The state transition with <eps> is a transition that gives a constraint that a predetermined word string (sentence) included in the document text set is continuously spoken. The parameter for the state transition with <eps> is expressed by (ε / ε: ω _u ). Here, ε before the slash indicates that no word is input in this transition. In addition, ε after the slash indicates that there is no output in this transition. Also, ω _u is one of transition weights, and by giving an appropriate weight (numerical value), the WFST can increase the score of sentences that match longer.

[3. WFST in this embodiment]
FIG. 10 is a schematic diagram showing an example of WFST used in this embodiment. The following description will focus on the difference between the general WFST described with reference to FIG. 9 and the WFST used in this embodiment.

  The WFST shown in FIG. 10 is also represented by a state transition diagram composed of nodes (states) and branches (state transitions). In the figure, nodes are represented by black circles or black square symbols, and branches are represented by lines with arrows. Each node is given a three-digit integer for identifying the node. Also in the state transition diagram shown in the figure, there are a start point node 601 and an end point node 608. In the state transition diagram shown as an example, there are two sequences in parallel from the start node 601 to the end node 608. The first sequence reaches from the node 601 at the start point to the node 608 at the end point through the nodes 602 to 607. Further, the second series reaches from the start node 601 to the end node 608 via the nodes 612 to 617. Each of these series corresponds to a sentence in the manuscript text. Here, only two sentences are shown for the sake of simplicity, but in reality there is no restriction on the number of sentences. A branch (the above-described epsilon transition) for returning from the end node 608 to the start node 601 exists.

Each state transition is defined by a transition source state, a transition destination state, an input symbol (word), an output symbol (word string), and a transition weight value. As an example, in the figure, the state transition from node 602 to node 603, an input word to accept a is W _2, the symbol to be output is epsilon (i.e., the output word is not). As another example, in the figure, the state transition from node 614 to node 615 is the symbol for receiving the epsilon (i.e. no word to accept), symbols output from a word string of C _3.

  In the figure, the display of the weight of state transition is omitted. Further, in the figure, a transition for accepting a speech recognition error is omitted. That is, the state transition for accepting a replacement word, the state transition for accepting an insertion word, and the state transition for allowing omission are omitted.

A feature of the WFST shown in the figure is that a document sentence is divided into chunks, and output transitions are arranged as state transitions at chunk breaks. The output transition is a state transition in which the output symbol is not empty. That is, the word string may be output not only at the end of the sentence but also at a state transition in the middle of the sentence. As a specific example in the figure, an original sentence (W ₁ , W ₂ , W ₃ , W ₄ , W ₅ ) is divided into a plurality of chunks of o ₁ and o ₂ . Chunk o ₁ is a word string (W ₁ , W ₂ , W ₃ ), and chunk o ₂ is a word string (W ₄ , W ₅ ). An output transition in which the input symbol is ε (that is, there is no input word) and the output symbol is C ₁ is provided between the chunks o ₁ and o ₂ . The state transition from node 607 to node 608 is the output transitions at the end of the sentence, the output symbol is C _2. Similarly, the original text (W ₆ , W ₇ , W ₈ , W ₉ , W ₁₀ ) is also divided into a plurality of chunks o ₃ (not shown) and o ₄ (not shown). Chunk o ₃ is a word string (W ₆ , W ₇ , W ₈ ), and chunk o ₄ is a word string (W ₉ , W ₁₀ ). Then, after the chunk o ₃ , an output transition for outputting the word string C ₃ is provided. Further, after the chunk o ₄ , an output transition for outputting the word string C ₄ is provided.

  As described above, in this embodiment, a sentence is divided into chunks and an output transition is provided at the end of the chunk. The method of dividing the sentence into chunks is arbitrary. Divide into chunks at grammatically or semantically breakable points in the sentence (for example, syntactic phrases can be used), or mechanically divide into multiple chunks with a predetermined number of words May be. Moreover, you may divide | segment in the location where the reporter or the announcer has started a new line for readability. Further, it may be divided into chunks of an appropriate length according to the degree of redundancy of sentences in the original and the required recognition error correction capability. Further, it may be divided into chunks of an appropriate length according to the allowable subtitle display delay. When a long word string is set as a chunk, the correction accuracy is improved, but the determination of the subtitle word string to be transmitted is delayed. If a short word string is set as a chunk, the subtitle word string to be sent can be determined quickly, but the correction accuracy is reduced. Further, division into chunks of an appropriate length may be performed according to the expected recognition accuracy of voice recognition and the degree of matching between the original and the reading voice.

[4. WFST construction method]
Next, a WFST construction method by the transducer construction device 740 will be described. When the transducer construction device 740 constructs a WFST, the original text included in the original text storage device 700 is divided into sentences in advance. If the end of a sentence is delimited by a punctuation mark, the punctuation mark can be divided into landmarks. Further, by analyzing the text of the manuscript, sentence breaks may be detected from the syntax rules and divided into sentences. Further, it may be possible to divide into sentences at places marked manually.

  In this embodiment, the text stored in the manuscript text storage device 700 is a reading manuscript that may be picked up by a news program. The manuscript prepared here includes manuscripts for a plurality of news items. Each news item includes a plurality of sentences. In this embodiment, it is not necessary to specify in advance the order in which news items are read or the order in which sentences are read. All documents that may be read may be prepared and stored in the document text storage device 700. In addition, among these manuscripts, there may be news items and sentences that cannot be read as a result.

  Prior to the construction of the WFST, the manuscript is previously shaped manually. By this shaping, word strings (sentences) that are always read continuously are collected as one sentence unit. Also, by this shaping, a portion that may be skipped in the document is separated as another sentence.

The sentence is divided into chunks in advance. If the chunk (that is, the unit of the word string until the output transition is performed) is set long, the correction accuracy increases, but the determination of the word string to be output is delayed. On the other hand, if the chunk is set to be short, the determination of the subtitle word string to be sent out becomes fast, but the correction accuracy decreases. Therefore, what unit should be used may be appropriately designed according to the expected recognition accuracy of voice recognition and the degree of matching between the original and the reading voice. Whatever method is used to divide a sentence into chunks, a chunk is a shorter unit than a sentence.
The transducer construction device 740 then configures the WFST to place the output transition immediately after receiving the chunk.

A specific WFST construction procedure by the transducer construction device 740 is as follows. The construction of the WFST is started from the start point of the WFST.
Each time the transducer construction device 740 reads one word at a time from the document text storage device 700, it sequentially creates a transition with a weight of 0 and a new node for accepting the word. Here, the transition of weight 0 means (s / ε: 0.0) when expressed by a triple parameter. Note that s is a word. Then, after adding an output transition after the chunk as a unit, the same processing is repeated for each word included in the next chunk. When the end of the chunk is the end of the sentence, the output transition related to the last chunk is connected to the end point node of WFST. That is, the transition destination state of the output transition related to the last chunk is set as the end node.
If there are more sentences left, start again from the starting point and repeat the above process.
Thereafter, the same processing is repeated until the processing for all sentences included in the document text storage device 700 is completed.

After reading all the original texts from the original text storage device 700, the transducer construction device 740 next connects the end point node and the start point node with an epsilon transition. As already described, the epsilon transition is a state transition of (ε / ε: ω _u ) in terms of a triple parameter. The transducer construction device 740 gives an appropriate value as the weight ω _u for this epsilon transition. As a result, the WFST can increase the score of sentences that match longer. This also allows the WFST to operate properly even when a sentence that matches the prefix of another sentence exists in the document. Finally, the transducer construction device 740 adds a transition that accepts substitution, omission, and insertion to the transition of each word. In FIG. 10, state transitions for replacement, omission, and insertion are omitted.

[5. Adaptation of speech recognition]
When the speech recognition device 720 actually recognizes the speech read out based on the original, it is desirable to adapt the language model for speech recognition based on the word string included in the original in advance. As a result, it is possible to increase the accuracy of voice recognition when the text is read as it is.

[6. Traceback from output transition]
In this embodiment, the node that starts the traceback is limited to the node immediately before the output transition. That is, in the WFST state transition diagram illustrated in FIG. 10, only the nodes indicated by black square symbols are the target of the traceback. The nodes indicated by the black circle symbols are not subject to traceback. In other words, the node that is the target of traceback is only the node corresponding to the end of each chunk.

Compared to performing traceback for all nodes of WFST from the maximum likelihood node (the node having the highest score among all the nodes at time t), the above method of this embodiment reduces the output delay. It can be shortened. That is, in this embodiment, it is possible to reduce the delay in displaying the corrected caption according to the result of the speech recognition process.
In the WFST construction method already described, a sentence is divided into chunks in advance, and an output transition is provided following the end of each chunk. However, it is also possible not to divide the sentence into chunks (in other words, one sentence as one chunk) and to limit the node that starts traceback to only the node immediately before the output transition. In such a case, the output delay can be shortened as compared with the case where the traceback is performed from the maximum likelihood node for all nodes.

[7. How to confirm subtitle text output]
A method for the speech recognition error correction apparatus 600 to determine a word string to be output will be described.
N (tilde) (t), which is the maximum likelihood node immediately before the output transition at time t (in other words, the maximum likelihood node in the node corresponding to the end of the chunk), is expressed by the following equation (1).

In Expression (1), Q is a set of nodes that are subject to traceback. That is, Q is a set of nodes immediately before the output transition. n is a node belonging to the set Q. L _n (t) is the score of node n at time t. The score L _n (t) is expressed by the following formula (2).

In Expression (2), e is a state transition, and ^ef and ^et are the transition source and transition destination nodes of the state transition e, respectively. E ^w is the value of the state transition weight of the state transition e. E ⁱ is an input symbol of the state transition e. E ⁱ is one of the words W, ε (empty input), and * (wildcard that matches any input). That is, as shown in Expression (2), the score (likelihood) of the node n at the time t is the transition of the state transition e at the time t−1 among all the state transitions with the node n as the transition destination node. the score of the source node e ^f, is obtained by adding the weight value e ^w a state transition e.

In order to obtain the maximum likelihood node according to the above equation (1), the speech recognition error correction apparatus 600 performs an efficient Viterbi search for the score L _n (t) while performing pruning with a threshold.

Then, the speech recognition error correction apparatus 600 sequentially traces back the state transitions e passed until reaching the maximum likelihood node obtained by the equation (1) to obtain a state transition sequence. Already it results traced back to the time t-t'of already output word h _t-t'is expressed by the following equation (3).

In Equation (3), {e _{t−t ′} ,...} Is a set of state transitions that do not include the start of WFST. Also, {..., E _t } is a set of state transitions that do not include the end of WFST. O (tilde) _t is a sequence of a section l ^t _k (that is, l ^t _k is a sentence) sandwiched between the start and end, as expressed by the following expression (4) (k = 1, 2, ...).

In Expression (4), r ^t is the number of sections l ^t _k (sentences) included in O (tilde) _t . Furthermore, this l ^t _k is a sequence of chunks, as shown in equation (5) below.

In equation (5), m _{t, k} is the number of chunks included in the interval l ^t _k .
The speech recognition error correction apparatus 600 outputs a chunk sequence represented by the following equation (6) for each section l ^t _k . That is, what is expressed by the following formula (6) is a caption sentence output by the speech recognition error correction apparatus 600.

In Equation (6), E (tilde) (o ^{t, k} _u ) is an error score based on the error rate. T is a threshold value regarding the error score, and 0 <T <1. The error score E (tilde) (o ^{t, k} _u ) is obtained by the following equation (7).

In the equation (7), the two-input function E (tilde) (o ₁ , o ₂ ) is expressed by E (tilde) (o ₁ ) and E (tilde) (o ₂ ) as shown in the following equation (8). ) Of the word weighted average.

In Expression (8), N ₁ and N ₂ are the numbers of words included in chunks o ₁ and o ₂ , respectively. E (tilde) (o) is an error rate (edit distance) obtained from the number of accepted words N _r in chunk o and the number of transitions N _e due to acceptance of an error. 9).

That is, as shown in the equation (7), E (tilde) (o ^{t, k} _u ) ^, which is an error score of the chunks o ^{t, k} _u included in the interval l ^t _k , is defined as follows. . That is, the error E ^{(tilde) (o t,} _{k u),} the chunk ^{o t,} and the error rate calculated by Equation (9) in the _{k u,} the previous chunk ^{o _t,} is calculated in the _{k u-1} Based on the score and the weighted average value by number of words. However, when the weighted average value based on the number of words is smaller than a predetermined threshold value T, the value of E (tilde) (o ^{t, k} _u ) is zero (the upper case in the right side of Expression (7)). ). On the other hand, when the weighted average value based on the number of words is equal to or greater than the threshold value T, the weighted average value is directly used as the value of E (tilde) (o ^{t, k} _u ) (lower right side of Expression (7)). Case). In this way, the value of E ^{(tilde) (o t,} _{k ^u)} is dependent on the value of the previous chunk ^{o _t,} E for _{k u-1 ^{(tilde) (o t, k u-}} 1) The output can be determined by calculating recursively using equation (7).

In addition, when the error score in the previous chunk is small by cutting off the error score when the value is lower than the threshold T to 0 by dividing into cases in Expression (7), The error score is not spread to the chunk.
Note that the specific value of the threshold T shown in the equations (6) and (7) may be determined as appropriate. For example, T = 0.5, or 0.4 ≦ T ≦ 0.6. It is preferable to set the value within the range. If the value of T is too large (approaching 1), a high error rate is allowed. On the other hand, if the value of T is too small, it becomes too strict against errors, and there is a disadvantage that the density of subtitles that can be output for a document that is originally read out becomes too low. Therefore, the range of 0.4 ≦ T ≦ 0.6 is preferable, and T = 0.5 is particularly preferable.

[8. Configuration of voice recognition error correction device]
Next, the configuration of the speech recognition error correction apparatus will be described.
FIG. 11 is a functional block diagram showing a schematic functional configuration of the speech recognition error correction apparatus 600. As illustrated, the speech recognition error correction apparatus 600 includes a WFST storage unit 610, a node data update unit 620, a node data storage unit 630, a document search unit 640, and an output unit 650.

  A WFST storage unit 610 (a finite state transducer information storage unit) is a finite state transducer that performs state transition while sequentially receiving words input as speech recognition results corresponding to a document text. Information on the state of a finite state transducer that makes a state transition while accepting an error in a word included in the result, and includes a transition source state, a transition destination state, an input symbol, an output symbol, and a transition weight regarding the state transition. Information is stored. The WFST storage unit 610 stores an output transition where the output symbol is a non-empty state transition as a part of the state transition. Also, information on the state and information on the state transition of the WFST in which a sentence included in the manuscript text is divided into a plurality of chunks and an output transition is provided after a state transition path that accepts a word included in the chunk as an input symbol. Remember.

  Specifically, the WFST storage unit 610 stores the WFST constructed by the transducer construction device 740 in advance based on the document text storage device 700 (FIG. 8). Specifically, the WFST storage unit 610 stores information equivalent to the state transition diagram (network). The information includes state (node) identification information and state attribute information, state transition (branch) identification information and state transition attribute information. The state transition attribute information includes at least transition source state identification information, transition destination state identification information, and a weight value related to the transition.

The node data updating unit 620 receives an input of a recognized word that is a speech recognition result corresponding to the document text from the outside, and according to the received recognized word, a score at the time (that time) for each state in the WFST. And the node data storage unit 630 is updated using the calculated score.
Specifically, the node data update unit 620 calculates, as node data, a score of a state that can be transited on the WFST network at each time when an input of a word included in the recognized word string output by the speech recognition device 720 is received. And to update. For example, each time one word is received from the speech recognition device 720, the node data update unit 620 sequentially performs a Viterbi search with reference to the WFST stored in the WFST storage unit 610, and stores it in the node data storage unit 630. Update the node data.

  The node data updating unit 620 calculates a score when the state transition is performed according to the word input as a part of the recognized word string. The node data update unit 620 calculates a score based on the value of the state transition weight. As described in the example of WFST, when words matching the corresponding original are input in the order of the original, “0” is added to the score as a weight value. When a word different from the corresponding manuscript is input, “−1” is added to the score as a weight value, and “−1” has a penalty meaning.

  For example, in the example shown in FIG. 9, when the word string input as the recognition word string is exactly the same word string as the corresponding manuscript, the word “Last Month” is received from the start node 001 to correspond to the word of the corresponding manuscript. Go to node 002 through the transition to. Then, in response to this state transition, the node data update unit 620 adds “0” to the score. Next, for example, the word “no” is received from the node 002 and the process proceeds to the node 003. Then, in response to this state transition, the node data update unit 620 adds “0” to the score. Similarly, when “Kanto Koshin”,... Are sequentially received, “0” that is the value of the weight of state transition corresponding to each word is added to the score.

  On the other hand, for example, in the example shown in FIG. 9, when the word string input as the recognition word string is a word string different from the corresponding document, the state transition is different. In other words, when the word “Last week” is received from the start point node 001, for example, the word “Last month” of the corresponding document is replaced, so the process proceeds to node 002 through a transition corresponding to the replacement. In this case, the node data update unit 620 adds “−1” (penalty) that is the value of the weight of the state transition to the score. Similarly, the node data update unit 620 adds a penalty “−1” to the score even when a transition corresponding to an insertion error or a drop error is passed.

  Thus, when the input recognition word is the same as the word s accepted in the state transition in WFST, the score in the transition is the best. On the other hand, state transitions corresponding to editing of replacement, insertion, and deletion are factors that deteriorate the score. Here, “edit” refers to a result obtained by replacing or deleting a word that should be originally inserted, or by inserting a word that is not a word that should originally be. For example, a state transition with <D> can be transitioned even if there is no matching input word. However, in the case of a path that passes only a transition with <D>, the penalty increases as the number of transitions increases. And the score goes down. WFST produces a result that if the recognition word string includes an error or paraphrase, the score is deteriorated accordingly.

  The node data storage unit 630 stores a score representing the likelihood of a state in WFST. That is, the node data storage unit 630 stores node data calculated by the node data update unit 620. The node data update unit 620 updates the node data storage unit 630 in a timely manner based on the calculation result described above. The node data storage unit 630 is configured using a general storage unit such as a semiconductor memory or a hard disk device, for example.

  When the document search unit 640 receives an activation signal indicating the start of processing from the outside, the document search unit 640 refers to the node data storage unit 630 to determine the maximum likelihood node at that time, and also includes the WFST storage unit 610 and the node data storage unit 630. And tracing back the state transition from the maximum likelihood node to perform a traceback process until a predetermined time when the state transition has been confirmed, and setting the path of the state transition subjected to the traceback process as an output candidate, For an output candidate path, an error score is calculated according to the ratio of the state transition related to the error in the path, and when the error degree is smaller than a predetermined threshold based on the calculated error score, The output candidate is determined output. Further, the manuscript search unit 640 calculates an error score for each chunk, and outputs an output symbol of an output transition corresponding to the chunk when the degree of error is smaller than a predetermined threshold based on the error score for each chunk. The document search unit 640 calculates the error score for each chunk according to the ratio of the state transition related to the error with respect to the chunk and the section of the chunk immediately before the chunk. The error score is calculated as the error score of the chunk.

  Further, the document search unit 640 determines the node having the maximum likelihood among the nodes corresponding to the transition source state of the output transition as the maximum likelihood node at that time. In other words, the document search unit 640 does not consider whether or not it is the maximum likelihood node for a state that is not the transition source state of the output transition.

  Specifically, the manuscript search unit 640 does not wait for input of recognition results of all recognition word strings for all manuscripts for determining the final best hypothesis, and each time the predetermined processing start condition is satisfied, the maximum likelihood The correction result is obtained. When the processing start condition is satisfied, the manuscript search unit 640 traces the final best hypothesis partially while tracing back on the WFST network based on the node data stored in the node data storage unit 630 at that time. Approximate hypotheses are sequentially determined as error correction results.

  The document search unit 640 approximates the final best hypothesis based on the edit distance between the word string of the corresponding document included in the WFST and the input recognition word string. The document search unit 640 determines that the path section is reliable if the edit distance in the path section from the beginning to the end is small to some extent between the paths divided for each predetermined range on the WFST network. And output. Here, the short edit distance means that the path through which the recognized word string and the original word string are almost matched has been passed. On the other hand, since the reliability of a path section with a long editing distance is low, it is not determined at that time and is used for the next traceback. It is presumed that a path section having a low reliability forever is a section talking about a difference that is not originally described in the manuscript. Therefore, a path section with low reliability is not output.

  The process start condition in the manuscript search unit 640 is, for example, a case where a silence period without speech is reached or a number of input words as a recognition word string output by the speech recognition device 720 is a predetermined word. This is the case when the number is reached. The predetermined period is not particularly limited, but is 3 seconds as an example. The predetermined number of words is not particularly limited, but is 20 words as an example. The activation signal input to the document search unit 640 is a signal indicating that the above process start condition is satisfied. The activation signal at this time may be automatically turned on, for example, by the voice recognition device 720 or may be turned on by a manual operation by the operator. The operator performs an operation of turning on the activation signal when recognizing a pause (silence of a predetermined length) or when recognizing a predetermined number of words output by the speech recognition device 720. When the speech recognition device 720 automatically turns on the activation signal, the activation signal is detected by detecting a pause using the result of the speech recognition or counting the number of words output as the speech recognition result. As a trigger to switch. With this configuration, the processing load can be reduced compared to the case where the search process is started each time a recognized word is input. If the silent period continues for a predetermined period, the recognition result word is not passed from the voice recognition device 720 during that period, so that the processing load of the device is not concentrated at that time, and the node score Can be easily compared.

  In order to realize the above function, the document search unit 640 further has the following configuration. That is, the document search unit 640 includes a maximum likelihood node detection unit 641, a traceback unit 642, a document division unit 643, an output candidate storage unit 644, an error score calculation unit 645, an error score determination unit 646, and a confirmation. An output storage unit 647 and a fixed time storage unit 648 are included.

  The maximum likelihood node detection unit 641 operates using an activation signal received from the outside as a trigger, and detects a node having the maximum score in the node data stored at that time. As described above, this activation signal is supplied from the outside when a predetermined processing start condition is satisfied.

  The traceback unit 642 follows the WFST network from downstream to upstream for the path from the node specified by the maximum likelihood node detection unit 641 to the node. At this time, the traceback unit 642 traces the network back to the place determined by the previous traceback. That is, the traceback unit 642 traces back to the time corresponding to the last input word of the word series that has been finalized and output.

  The document dividing unit 643 cuts out a word string for each section corresponding to the output transition in the traceback path. Here, the output transition is a state transition accompanied by the output of a symbol as described above. When the transducer construction device 740 constructs a WFST, an output transition is provided immediately after the end of the chunk. That is, the section obtained as a result of the division by the original dividing unit 643 is the above-described chunk unit. The document dividing unit 643 divides the entire path obtained as a result of the traceback, and outputs each word string obtained as a result of the division.

  The output candidate storage unit 644 stores the output transition output symbol (cut out document) corresponding to the path section obtained as a result of the division by the document dividing unit 643 as an output candidate. The output candidate storage unit 644 is configured using, for example, a general storage unit such as a semiconductor memory or a hard disk device.

  The error score calculation unit 645 calculates an error score for each section (chunk) divided and cut out by the document dividing unit 643. The error score calculation unit 645 calculates the error score using the already described formula (7).

  The error score determination unit 646 determines whether or not the error score calculated by the error score calculation unit 645 is smaller than a predetermined threshold T. In other words, the error score determination unit 646 determines based on the condition included in the already described formula (6).

  The confirmed output storage unit 647 stores an output (referred to as a confirmed output) that has been confirmed based on the determination result by the error score determination unit 646. Specifically, when the error score determination unit 646 determines that the error score is smaller than the threshold T, the confirmed output storage unit 647 stores the output symbol of the output transition in the predetermined path section as an error correction result. In other cases, the definite output storage unit 647 does not store the output symbol of the output transition in the predetermined path section as an error correction result. The confirmed output storage unit 647 is configured using a general storage unit such as a semiconductor memory or a hard disk device. The storage structure of the definite output storage unit 647 is a stack, and the definite output storage unit 647 holds data in a last-in first-out (LIFO) structure.

  The fixed time storage unit 648 stores the fixed time determined by the latest traceback process. The fixed time storage unit 648 is a general storage unit such as a semiconductor memory or a hard disk device. The fixed time storage unit 648 stores the latest path symbol corresponding to the output symbol loaded on the stack at the time when the determination process by the error score determination unit 646 is completed for all the path sections (all cut out documents) traced back this time. The time of the confirmed word is stored as the confirmed time.

  The output unit 650 sequentially outputs the corresponding document (that is, the confirmed output) determined as the error correction result by the document search unit 640. The output unit 650 is determined by the time when the determination process for the error score calculated for each path section of all the cut out corresponding documents in the path traced back through the WFST network is completed. Outputs output symbol data. Specifically, the output unit outputs the output symbol data stacked in the stack of the definite output storage unit 647 until the stack becomes empty.

  This speech recognition error correcting device 600 performs an operation of correcting and outputting a word that has been erroneously recognized, and not outputting a significantly incorrect recognition result. In other words, if the result of the correction by the speech recognition error correction device 600 can be seen in advance by a person, an error is enough to feel that “this is not a sentence” or “the meaning is different”. The operation in which the speech recognition error correction apparatus 600 detects the part in the process and does not output the detected part is included as error correction in a broad sense. This is because an error score is calculated and discrimination is performed based on the error score.

[9. Operation procedure of speech recognition error correction device]
FIG. 12 is a flowchart showing a processing procedure performed by the speech recognition error correction apparatus 600. The premise of the processing of this flowchart is the following four points.
(Assuming 1) the recognition result of a word input _{{ω 0, ω 1, ...} , ω k, ..., ω j, ...} and.
(Premise 2) Let ω _k be the last input word of the portion determined by the previous traceback, and let the output transition at that time be a _P (P-th output transition along the time axis).
(Premise 3) Consider a case in which, after a word ω _{j as a} recognition result is input, a predetermined silence is used as a trigger for successive determination.
(Premise 4) The node data updating unit 620 accepts the last input word ω _j before silence and calculates all the nodes that can make a transition.

  Hereinafter, the flow of processing by the speech recognition error correction apparatus 600 will be described with reference to this flowchart.

  First, in step S501, using the input of an activation signal from the outside as a trigger, the maximum likelihood node detection unit 641 detects the node with the highest score in the node data stored at that time as the maximum likelihood node. The state represented by this detection node is the maximum likelihood state at the start of traceback.

Next, in step S502, the traceback unit 642 traces back from the maximum likelihood node detected in step S501 to the time determined in the previous traceback process. Specifically, the traceback unit 642 traces the word history on the WFST in the reverse direction (in the direction of going back in time) for the path that has reached the detected maximum likelihood node, and confirms and outputs it in the previous traceback. Find the last input word ω _k of the word sequence. Further, the traceback unit 642 obtains a corresponding state transition (a state transition in which the accepted word is ω _k ). Then, trace back is performed until a fixed time corresponding to the word and the state transition. The traceback unit 642 refers to the confirmed time storage unit 648 and sets the confirmed time stored in the confirmed time storage unit 648 as the confirmed time corresponding to the last input word ω _k described above.
Note that instead of the state transition in which the accepted word is ω _k , the traceback may be performed until the output transition a _P is reached.

In step S503, the document dividing unit 643 divides the document and sets word strings obtained as a result of the division as output candidates. Specifically, the document dividing unit 643 divides the document for each path section sandwiched between two output transitions in the path traced back this time. The document dividing unit 643 stores the obtained output candidates in the output candidate storage unit 644.
Note that, as a processing method here, the output transition a _L (Lth (but L) along the time axis) that can be output while proceeding in the reverse direction (that is, in the direction of going back in time) until the output transition a _P is reached. The document may be divided every time it passes (> P) output transition). It is also possible to divide the original each time passing through the output transition a _L can be output from the side of the output transition a _P.
Note that the output transition a _L that can be output is an output transition symbol that is an output candidate, but is output later that is rejected by the error score determination unit 646 (that is, not output from the output unit 650 after all). Output transition).

Next, in step S504, the error score calculation unit 645 calculates an error score of the output candidate. Specifically, the error score calculation unit 645 calculates the equation (7) by the chunk ^{o t,} _{k u} of the error score E ^{(tilde) (o t,} _{k u).}

  In step S505, the error score determination unit 646 selects one of the output candidates and determines whether the error score calculated for the output candidate is less than the threshold T. If the error score is less than the threshold value T (step S505: YES), the process proceeds to the next step S506. If the error score is greater than or equal to the threshold T (step S505: NO), the process jumps to step S508.

When the process proceeds to step S506, the error score determination unit 646 determines the output transition of the path section (chunk) on the WFST network, and determines the output symbol as an error correction result.
In step S507, the error score determination unit 646 writes the data of the output symbol determined this time in the determined output storage unit 647. As described above, the definite output storage unit 647 is a stack having a last-in first-out structure.
The processes in steps S506 and S507 are executed only when the error score of the current chunk is less than the threshold T by the branch control in step S505.

  Next, in step S508, the error score determination unit 646 determines whether all output candidates have been determined. If discrimination of all output candidates has been completed (step S508: YES), the process proceeds to the next step S509. If discrimination of all output candidates is not completed, that is, if there are still output candidates to be selected (step S508: NO), the process returns to step S505 to select the next output candidate.

That is, when the determination process for the error score calculated for each path section corresponding to all the cut out originals is completed, the process proceeds to the next step S509.
In step S509, the error score determination unit 646 updates the confirmed time storage unit 648. That is, the error score determination unit 646 stores, in the determination time storage unit 648, the determination time determined by the current traceback process as the determination time determined by the current traceback process, with the latest determination word corresponding to the output symbol stacked on the stack.

  In step S <b> 510, the output unit 650 sequentially reads and outputs the output symbol data currently written in the determined output storage unit 647 until the determined output storage unit 647 becomes empty. As a result, all the output data loaded on the stack is output. Note that the output unit 650 outputs in order from the document placed on the front side of the WFST network (that is, in order from the earliest time).

  That is, the output unit 650 sequentially outputs the data accumulated on the stack in each traceback process as a confirmed document. At this time, if the error score of a predetermined path section is equal to or greater than the threshold T among the speech recognition results, the output symbol of the output transition of the path section is not adopted as the error correction result because the path has low reliability. Therefore, the output unit 650 does not output such output symbols in the path section with low reliability.

  When the process of step S10 ends, the speech recognition error correction apparatus 100 ends the process of the entire flowchart.

[10. Specific examples and evaluation of subtitle text output]
As described above, the features of the processing of the speech recognition error correction apparatus 600 according to this embodiment are (1) dividing a sentence into chunks, (2) providing an output transition at the end of the chunk, and (3) output. Trace back from the transition point. The effect | action by performing such a characteristic process is demonstrated below with a specific process example.

  FIG. 13 is a schematic diagram illustrating an example of a subtitle sentence output confirmation method according to this embodiment. (A) and (B) in the figure respectively show the relationship between the input recognition result word string, the WFST state transition corresponding to those words, and the resulting output symbols, and the calculation. The error score to be added is added.

  First, in the example of FIG. 5A, the input word string is “today / no / kanto / region / ha”. When these words are accepted, the WFST transitions as follows. That is, starting from the beginning, the input word “today” matches “today” in the corresponding document, and a state transition without penalty occurs. The next word “NO” matches “NO” in the corresponding manuscript, and a state transition without a penalty occurs. The next word “Kanto” matches “Kanto” in the corresponding manuscript and a state transition without penalty occurs. The next word “region” is a word that is input due to a replacement by the speaker or a recognition error by the speech recognition device 720. Therefore, the “region” in the corresponding manuscript does not match and a replacement state transition occurs. The next word “ha” matches “ha” in the corresponding document, and a state transition without penalty occurs. “Sunny” and “Is” are not input, but when a dropout state transition occurs, the end state of the chunk indicated by the black square is reached. The output symbol “Today's Kanto region is clear”, which is an output symbol in the output transition, is an output candidate. Here, the error rate for this chunk is 3/7. That is, the error score is 3/7. When the threshold value T is 0.5, this error score is smaller than the threshold value T. Then, the speech recognition error correction apparatus 600 can determine the output of “Today's Kanto region is sunny”, which is an output candidate.

  Next, in the WFST network shown in the example of FIG. 5B, a series of sections is divided into abcdef chunks and ghijk chunks. An output transition is provided at the end of each chunk. The input word string is abcxeyegh. When these words are accepted, the WFST transitions as follows. That is, starting from the beginning, each of the input words a, b, and c matches a, b, and c in the corresponding manuscript, and a state transition without penalty for each of these words occurs. For the next input word x, a state transition of replacement of the word d in the document occurs. The next input word e matches e in the manuscript, and a state transition without penalty occurs. Then, for the next input word y, the state transition of replacement of the word f in the document occurs. The transition destination is the end state of the chunk, and the output symbol in the next output transition is abcdef. The chunk error score so far is 2/6 (because there are 4 normal state transitions and 2 replacement state transitions, 2 / (4 + 2)).

  Subsequently, in FIG. 5B, each of the input words g and h matches g and h in the corresponding manuscript, and a state transition without penalty for each of these words occurs. Here, the input word string ends, but o, j, k in the corresponding manuscript may be dropped. Here, the apology rate is only 3/5 for the section of the ghijk column in the document, which exceeds the threshold T (= 0.5). This is because g, h match the input and i, j, k are missing. However, the error score calculated by the equation (9) is 5/11 by taking a weighted average (weighted average based on the number of words in the chunk) with the error score in the previous chunk. This error score 5/11 is smaller than the threshold value T. Therefore, the speech recognition error correction apparatus 100 outputs the output candidate abcdef and also outputs the output candidate ghijk corresponding to the next chunk.

As shown in the example of FIG. 5A, the speech recognition error correction apparatus 600 can detect the input word after the word “ha” all at once (eg, recognition error by the speech recognition apparatus 720). The corresponding subtitle sentence can be output. This is because the error score is smaller than the threshold T even if all input words after the word “ha” are incorrect.
Further, as shown in the example of FIG. 5B, when the chunk is divided in advance, the error score of the chunk that has been confirmed ahead of the currently focused chunk is obtained by the action of the equation (9). By using this, it is possible to determine the output of the currently focused chunk even earlier.

[11. How to determine threshold T for error score]
If the recognition accuracy of the speech recognition apparatus is about 90%, the error rate of the word that is the basis of the error score may be about 10%. In this embodiment, the threshold T used for determination by the error score determination unit 646 is preferably set with a margin corresponding to the reliability of the word matching rate according to the recognition accuracy of speech recognition. Here, the reliability of the word match rate depends on the number of words between two output transitions of the WFST network.

As another factor for setting the threshold value T, there is a ratio of the overlapping of candidate document sentences stored in the document text storage device 700 as a percentage. For example, in the case of sentences shown in (E1) to (E3) below, overlapping as sentences is included at a rate of about 80%.
(E1) Today's weather is sunny (E2) Today's weather is rainy (E3) Today's weather is cloudy In this case, if the threshold T is set to about 80%, the desired movement will be realized Can not.

  It was confirmed that when the test was performed under the condition that the sentence of the news manuscript was appropriately divided into chunks and the output transitions were arranged and the threshold T was 50%, it was confirmed that the operation was good.

  As described above, the speech recognition error correction apparatus 600 according to the present embodiment uses the constraint that the sentences included in the manuscript text are continuously uttered in principle, although the order of the sentences can be changed. The recognition error can be corrected by associating the recognition result with the original text. That is, the automatic correction error of the block matching method according to the prior art is solved.

  Also, the speech recognition error correction apparatus 600 according to the present embodiment collates where the word string of the recognized word most closely matches the word string included in the manuscript text within a longer range than the conventional technique. Yes. That is, while the conventional block matching method collates only the section corresponding to the word chain block, the speech recognition error correction apparatus 600 collates the whole sentence while tracing back the sentence of the document. Therefore, it is clearly known where to make the match, and automatic correction errors can be reduced more than before.

  The speech recognition error correction apparatus 600 according to this embodiment divides a sentence into a plurality of chunks and provides an output transition at the end of the chunk. As a result, the confirmed word string can be output even in the middle of the sentence.

  Further, the speech recognition error correction apparatus 600 according to the present embodiment is limited to trace back only from a place where an output transition exists. Thereby, the amount of calculation can be suppressed.

  Although the presupposed form has been described above, the present embodiment can also be implemented in the following modified example. Moreover, you may implement combining a some modification.

[Modification 1] Configuration of Speech Recognition Error Correction Device The speech recognition error correction device 600 may include a transducer construction device 740 inside.

[Modification 2] Construction of WFST that accepts paraphrasing An announcement manuscript that is an information source of WFST includes phrases that are skipped when read, paraphrased phrases, and supplemented phrases There is a case. Some of these are regularly omitted and paraphrased / supplemented with high frequency. For example, in a news program manuscript, phrases such as “according to the Metropolitan Police Department” representing the interview source are often skipped. Even if such a phrase is skipped, there is no change in the meaning of the main text of the news, in other words, there is no change regarding so-called 5W1H, and there is no practical problem as a news program.

  In the present modification, such a standard phrase variation is added to the WFST so that the correction result can be output with high accuracy. The addition of the above-mentioned wording variations is performed efficiently by constructing another WFST for adding wording variations separately from the WFST constructed from the manuscript and combining it with the WFST constructed from the manuscript. Can do. A known algorithm for performing WFST synthesis, minimization, determinization, or the like can be applied.

  In order to construct a WFST including the above paraphrasing example, the difference between the original text of the same type of program in the past and the word string actually read out is analyzed, and the frequency is high and the meaning of the text is determined by paraphrasing. Select and collect items that do not change in advance. Then, for each selected paraphrase example, a WFST for synthesizing the paraphrase is constructed, and an operation (computation based on the existing technology) for synthesizing the WFST constructed from the manuscript and the WFST of the paraphrase example is performed. In other words, a WFST that can be used in other words can be constructed.

[Modification 3] Addition to Recognition Result Symbols that cannot be obtained as a result of speech recognition (punctuation marks, reading marks, other symbols, etc.) may be included in the output symbols in the output state transition. For example, those symbols and the like are previously included in the news manuscript, and when the WFST is constructed based on the news manuscript, the symbols and the like remain in the output symbol. By operating the speech recognition error correction apparatus 100 using such WFST, it is possible to output easy-to-read subtitles including those symbols and the like.

[Modification 4] Other output corresponding to the recognition result (multilingual subtitles)
In addition to the third modification described above, a word string obtained as a result of translating the text of a document into another language or the like may be used as an output symbol in output transition. Thereby, it is possible to output subtitles in a language different from that of the original document read out. Also, subtitles in a plurality of languages can be output by operating a plurality of WFSTs in parallel. Further, a symbol corresponding to a queue for controlling the progress of a program may be included as an output symbol in the output transition. With the output of this queue as a trigger, it is possible to give instruction information such as start-up and scene change to a program-linked service such as hybrid cast. As a result, more various broadcasting services can be realized.

[Modification 5] Minimization of WFST In this modification, when creating a WFST, the WFST is minimized if possible. The minimization of WFST is to aggregate a plurality of states (nodes) that can be aggregated or to aggregate a plurality of state transitions (branches) that can be aggregated in a given state transition diagram. Aggregating WFST itself can be performed by existing technology. An example of WFST minimization is as follows. That is, in the WFST network, a plurality of common partial networks are reconfigured as the same state transition sequence. By using such WFST minimization, state transitions for common word strings can be reduced. For example, word strings (sentences) having the same prefix can be shared by the same transition. Thereby, the amount of calculation can be reduced.

[Modification 6] Determining the WFST When creating the WFST, the WFST is determinized if necessary. Specifically, in the WFST network, when an output symbol is determined during the state transition, the position of the output sentence is changed to the front in order to output the estimation result as soon as possible. By determinizing the WFST, for example, the output sentence is moved to a transition with a unique prefix. This has the advantage that the output sentence can be finalized in the long term. However, if the WFST is determinized when creating the WFST, it is necessary to change the setting so that the search process of the maximum likelihood hypothesis by the manuscript search unit 640 can also cope. In other words, it is necessary to shift the path interval for calculating the error score before and after the output transition as compared with the case where WFST is not determinized. In addition, it is necessary to set the threshold T to a stricter value (a value with a smaller error rate) so that the expansion and contraction of the preceding and following path sections can be absorbed.

[Modification 7] Method of obtaining error score In this embodiment, the error score is calculated based on the error rate (edit distance) shown in Equation (9). However, the present invention is not limited to this, and the error score may be calculated by using the matching rate, matching accuracy, dropout rate, and insertion rate of the document and the recognition result, or using them together.

The configuration of the method 2 described above is organized as follows.
[2-1] A finite state transducer that sequentially transitions while accepting words input as speech recognition results corresponding to the original text, and also includes errors in the words included in the speech recognition results. The finite state that stores information about the state of the finite state transducer that undergoes state transition while accepting, and information including a transition source state, a transition destination state, an input symbol, an output symbol, and a transition weight regarding the state transition. A transducer information storage unit; a node data storage unit that stores a score representing the likelihood of the state in the finite state transducer; and an input of a recognition word that is a speech recognition result corresponding to the manuscript text from outside And the state of the finite state transducer according to the received recognition word. A node data update unit that updates the node data storage unit using the calculated score and an activation signal indicating the start of processing from the outside, By referring to the maximum likelihood node at that time, and referring to the finite state transducer information storage unit and the node data storage unit, the state transition to the maximum likelihood node is traced back to Trace back processing is performed until a predetermined time when the transition is confirmed, and the path of the state transition subjected to the trace back processing is set as an output candidate. An error score corresponding to the included ratio is calculated, and the degree of error is determined based on the calculated error score. A speech recognition error correction apparatus comprising: a document search unit that sets a candidate for output of the path as a definite output when the value is smaller than a value; and an output unit that outputs the definite output obtained by the document search unit. The finite state transducer information storage unit stores an output transition where the output symbol is a non-empty state transition as a part of the state transition, and stores a sentence included in the original text in a plurality of chunks. Storing information relating to the state and information relating to the state transition of the finite state transducer provided with the output transition after a state transition path in which the word included in the chunk is received as an input symbol. A speech recognition error correction device characterized by the above.

  [2-2] The document search unit calculates the error score for each chunk, and responds to the chunk when the degree of error is smaller than a predetermined threshold based on the error score for each chunk. The output symbol of the output transition is used as the definite output, and the document search unit calculates the error score for each chunk with respect to the chunk and a section of the chunk immediately before the chunk. The speech recognition error correction apparatus according to [2-1], wherein an error score corresponding to a ratio of state transitions related to the error is calculated as an error score of the chunk.

  [2-3] A finite state transducer that sequentially accepts words input as a speech recognition result in response to a manuscript text and makes a state transition, and also includes an error in a word included in the speech recognition result. The finite state that stores information about the state of the finite state transducer that undergoes state transition while accepting, and information including a transition source state, a transition destination state, an input symbol, an output symbol, and a transition weight regarding the state transition. A transducer information storage unit; a node data storage unit that stores a score representing the likelihood of the state in the finite state transducer; and an input of a recognition word that is a speech recognition result corresponding to the manuscript text from outside And the state of the finite state transducer according to the received recognition word. A node data update unit that updates the node data storage unit using the calculated score and an activation signal indicating the start of processing from the outside, By referring to the maximum likelihood node at that time, and referring to the finite state transducer information storage unit and the node data storage unit, the state transition to the maximum likelihood node is traced back to Trace back processing is performed until a predetermined time when the transition is confirmed, and the path of the state transition subjected to the trace back processing is set as an output candidate. An error score corresponding to the included ratio is calculated, and the degree of error is determined based on the calculated error score. A speech recognition error correction apparatus comprising: a document search unit that sets a candidate for output of the path as a definite output when the value is smaller than a value; and an output unit that outputs the definite output obtained by the document search unit. The finite state transducer information storage unit stores an output transition where the output symbol is a non-empty state transition as a part of the state transition, and the document search unit stores the transition of the output transition. A speech recognition error correcting apparatus, wherein a node having the maximum likelihood among nodes corresponding to an original state is determined as the maximum likelihood node at that time.

  Next, a plurality of embodiments of the present invention will be described on the basis of the above-described speech recognition error correction apparatus (method 1 and method 2).

[First Embodiment]
The speech recognition error correction apparatus according to the first embodiment is a system in which a preprocessing unit having a function of preprocessing original text is further provided in the above-described speech recognition error correction apparatus of method 1.
FIG. 14 is a block diagram showing a schematic functional configuration of the speech recognition error correction apparatus according to the present embodiment. As shown in the figure, the speech recognition error correction apparatus 800 according to the present embodiment includes a preprocessing unit 811 in addition to the configuration provided in the speech recognition error correction apparatus 100 (method 1 of FIG. 1 and FIG. 3). Consists of including.

  The preprocessing unit 811 analyzes the data of the document text set 200 stored in the storage unit, and outputs warning information when a warning situation is detected. The warning situation is a situation in which manuscript text that is likely to cause an error in the operation of the manuscript search means 140 is included. In addition, the preprocessing unit 811 generates and outputs a correction plan for eliminating the warning situation. The operator performs an operation of correcting the manuscript text in accordance with the correction plan presented from the preprocessing unit 811 or referring to the correction plan.

  If the transducer construction device 240 constructs a transducer as it is based on the document text in a warning situation, an error may occur in the operation of the document search means 140, and the voice recognition error may not be corrected correctly. However, an error in the document search means 140 can be prevented by appropriately correcting the document text at the preprocessing stage by the processing of the preprocessing unit 811.

  Note that specific examples of warning situations and the processing of the preprocessing unit 811 for eliminating these warning situations will be described in detail later.

[Second Embodiment]
The speech recognition error correction apparatus according to the second embodiment is a system in which a pre-processing unit having a function of pre-processing original text is further provided in the above-described method 2 speech recognition error correction apparatus.
FIG. 15 is a block diagram showing a schematic functional configuration of the speech recognition error correction apparatus according to the present embodiment. As shown in the figure, the speech recognition error correction apparatus 900 according to the present embodiment has an original text storage device 700 in addition to the configuration provided in the speech recognition error correction apparatus 600 (method 2 in FIG. 11). (Original text storage unit), a transducer construction device 740, and a preprocessing unit 911.

  The pre-processing unit 911 analyzes document text data stored in the document text storage device 700, and outputs warning information when a warning situation is detected. The warning situation is a situation in which manuscript text that is likely to cause errors in the operation of the manuscript search unit 640 is included. In addition, the preprocessing unit 911 generates and outputs a correction plan for eliminating the warning situation. The operator performs an operation of correcting the manuscript text in accordance with the correction plan presented from the preprocessing unit 911 or with reference to the correction plan.

  If the transducer construction device 740 constructs the transducer as it is based on the document text in a warning situation, an error occurs in the operation of the document search unit 640, and the voice recognition error may not be corrected correctly. . However, an error in the document search unit 640 can be prevented by appropriately correcting the document text in the preprocessing stage by the processing of the preprocessing unit 911.

  Note that specific examples of warning situations and the processing of the preprocessing unit 911 for eliminating these warning situations will be described in detail later.

[Operation of the pre-processing unit (811 or 911) in the first embodiment and the second embodiment]
As described in the premise mode (method 1 and method 2), the speech recognition error correction device (800 or 900) determines and outputs a correction result that satisfies the following equation (10).

In Expression (10), O _t is a set of sentences or chunks that are candidates to be determined by traceback. O is a section belonging to O _t . E _o is an error rate obtained from the total of transition weights for accepting errors in the interval o. For example, E _o may be obtained from the number of passages of transitions that accept an error in the interval o. T is a threshold relating to the error rate. The threshold value T is based on the degree of mismatch between the accepted document and the recognition result, and the degree of coincidence between the utterance with no document (a word string not actually included in the document and actually recognized and recognized) Set as appropriate. For example, when T = about 0.5 (predetermined T satisfying 0.4 ≦ T ≦ 0.6), a good correction result of recognition error can be obtained. However, if there are pairs of similar sentences having a similarity of 0.5 or more (in other words, an edit distance of 0.5 or less) in the document, it is difficult to obtain an appropriate result.

  The preprocessing unit (811 or 911) calculates a distance representing a similarity between a plurality of sequences with respect to a sequence of language elements (words, characters, phonemes, etc.) included in the document text stored in the document text storage unit. In addition, a similar series is detected depending on whether the calculated distance is below a predetermined threshold, and warning information is output when a similar series is detected, and at least one of the detected similar series is output. By manipulating the manuscript text, the correction proposal for eliminating the similarity is output, and the manuscript text is amended according to the instruction inputted from the outside.

  In other words, the preprocessing unit (811 or 911) has a function of detecting a warning situation in the document in advance and issuing warning information. Further, the preprocessing unit (811 or 911) has a function of presenting a correction proposal regarding the found warning situation. The preprocessing unit (811 or 911) corrects the document based on the operator's operation after presenting the correction plan. In the most typical case, when the operator approves the amendment proposed by the preprocessing unit (811 or 911) and the operator performs an operation indicating the approval, the preprocessing unit (811 or 911) presents the amendment proposed. Follow the procedure to correct the manuscript. Further, it is possible to make a correction different from the correction proposal presented by the preprocessing unit (811 or 911) at the operator's discretion. In any case, the warning situation included in the document is resolved by the analysis and correction of the document by the preprocessing unit (811 or 911).

There are the following four specific examples of warning situations.
[1] Presence of duplicate items [2] Presence of similar sentences [3] Presence of short sentences (too short sentences) [4] Presence of similar chunks at the beginning of sentences Details of each of these warning situations will be described later.

Here, the configuration of the document is formally defined.
Let M be the entire document to be processed. The configuration of the document M is defined as the following formula (11).

In equation (11), d ₁ , d ₂ ,... Are news items. It should be noted that n _M is the number of news items that are contained within the original document M. That is, the document M is composed of n _M news items d _i (1 ≦ i ≦ n _M ).
The structure of the news item d _i is as shown in the following equation (12).

In the formula (12), l ₁ , l ₂ ,... Are sentences. In equation (12), n _i is the number of sentences included in the news item d _i . That is, the news item d _i is composed of n _i sentences l _j (1 ≦ j ≦ n _i ).
The structure of the sentence _{l j} is defined in Equation (13) below.

In Expression (13), c ^j ₁ , c ^j ₂ ,... Are chunks. In Expression (13), n _j is the number of chunks included in the sentence l _j . That is, the sentence l _j is composed of n _j chunks c ^j ₁ , c ^j ₂ ,.

That is, as described above, the structure of the manuscript text is as follows. That is, the entire document text is composed of one or more items. Here, the item is a news item in a news program, for example. An item is composed of one or more sentences. A normal sentence is a single group that forms a syntax according to the grammar of a language (such as Japanese). In Japanese, sentences are usually separated by punctuation. However, in the present embodiment, a plurality of grammatical sentences may be treated as equivalent to a single sentence group for convenience. For example, as described later, there are cases where sentences are connected. A sentence is composed of one or more chunks. A chunk is configured as a string of one or more words. A word consists of one or more characters. The “series” is a series of language elements. The items, sentences, and chunks described here are all series.
The manuscript text is stored in the storage means as data having such a logical structure. Further, the logical structure can be appropriately changed by the processing. Changing the logical structure means, for example, connecting sentences or changing the range of chunks.

[1. Detection of duplicate items by the preprocessing unit]
The first warning situation is that duplicate items exist.
For this reason, the pre-processing unit (811 or 911) outputs warning information indicating that the items are duplicated when the distance between the items is below a predetermined threshold. Specifically, it is as follows.
A word string obtained by concatenating all the word strings of the sentences of the news item d _i is d (hat) _i , and the set M (hat) is expressed by the following equation (14).

  Pre-processing unit (811 or 911) Using this set M (hat), a pair of items having a high degree of similarity is detected by the following equations (15) and (16). The preprocessing unit (811 or 911) calculates the distance between items in the detected pair.

In Expressions (15) and (16), x and y are word strings of news items that are elements of the set M (hat). L (x, y) is an edit distance between series. As the editing distance, for example, an improved version of Levenshtein distance is used. An improved version of this Levenstein distance will be described later.
D _d shown in Expression (15) is the minimum value of the mutual distance between the word strings (x, y) of the news items included in the set M (hat). Further, P _d shown in the equation (16) is a pair of word strings of news items having such a minimum distance.

Preprocessing unit (811 or 911) is in the set M, _{D d} is a predetermined threshold value (e.g., 0.5) when the below, and outputs a warning about news items pair _{P d} which forms the distance _{D d} . If the threshold setting is appropriate, this warning indicates that the same news item exists in the document. D _d factors that event such as below the above threshold occurs, a situation where errors news manuscript or organize news manuscript, is not completed. In other words, for example, when a correction is made to a manuscript sentence of a certain news item, the manuscript before and after the amendment remains in the set M.

Based on the above warning output from the preprocessing unit (811 or 911), the operator of the speech recognition error correction apparatus (800 or 900), for example, appropriate, such as by removing either one of the news item pairs P _d Take appropriate action.
Further, (i.e. updated) for even a new set M after treatment, the pre-processing unit (811 or 911) performs the same process to determine the _{D d} and _{P d.} When the D _d is below the threshold, as above, and outputs a warning.
Then, the preprocessing unit (811 or 911) is, D _d until no fall below the threshold value, repeating the detecting and eliminating duplicate items.

  FIG. 16 is a schematic diagram illustrating a first example of duplicate items detected by the preprocessing unit (811 or 911). In the figure, a pair of news items identified by “doc = 323” and “doc = 222” is detected as a duplicate item based on the distance between them. The item “doc = 323” includes four sentences from sentences 324 to 327. The item “doc = 222” includes seven sentences from sentences 223 to 229. In both items, the sentence pairs are in a similar relationship (including “similar” even if they are in the same relationship) as follows. That is, the similarity between the sentence 324 and the sentence 223 is high. Further, the similarity between the sentence 325 and the sentence 224 is high. Further, the similarity between the sentence 326 and the sentence 225 is high. Further, the similarity between the sentence 327 and the part after “invitation / promotion part / ha / 2020 / year / summer /” in the sentence 227 is high. In other words, considering the meaning, this item pair is considered to be a duplicate item representing the same news.

  FIG. 17 is a schematic diagram illustrating a second example of duplicate items detected by the preprocessing unit (811 or 911). In the figure, news items identified by “doc = 81”, “doc = 112”, and “doc = 99” are detected as duplicate items based on the distance between them. The item “doc = 81” includes three sentences 82 to 84. The item “doc = 112” includes three sentences 113 to 115. The item “doc = 99” includes three sentences 100 to 102. In these items, as described below, sentences are in a similar relationship (the case where they are in the same relationship is also included in “similar”). That is, the similarity between the sentence 82, the sentence 113, and the sentence 100 is high. Moreover, the similarity of the sentence 83, the sentence 114, and the sentence 101 is high. Moreover, the similarity of the sentence 84, the sentence 115, and the sentence 102 is high. Even considering the meaning of the text of each item, these three items are considered to be duplicate items representing the same news.

[2. Detection of similar sentences by the preprocessor]
The second warning situation is that a similar sentence exists in the document.
For this reason, the pre-processing unit (811 or 911) outputs warning information indicating that the sentences are similar when the distance between the sentences is below a predetermined threshold, and any of the similar sentences A suggestion for amendment that links to the sentence before or after the sentence is output. Specifically, it is as follows.

The preprocessing unit (811 or 911) detects similar sentences included in different news item pairs.
A word string formed by concatenating all the word strings of chunks included in the sentence l _j is defined as l (tilde) _j . M (tilde) is a set having such l (tilde) _j as an element. That is, the set M (tilde) is expressed by the following equation (17).

  The set M (tilde) has a word string corresponding to each sentence included in all news items as an element. Based on the set M (tilde), the preprocessing unit (811 or 911) detects sentence pairs having a high degree of similarity using the following expressions (18) and (19). Further, the preprocessing unit (811 or 911) obtains the distance between sentences in the detected sentence pair.

  In Expressions (18) and (19), x and y are word strings of sentences that are elements of the set M (tilde). L (x, y) is an edit distance between sequences, and an improved version of the Levenshtein distance is used as an example.

D ₁ shown in Expression (18) is the minimum value of the mutual distance between word strings (x, y) of sentences included in the set M (tilde). Further, _Pl shown in Equation (19) is a sentence pair that forms such a minimum distance.

Preprocessing unit (811 or 911) is, _{D l} is a predetermined threshold value (e.g., 0.5) when the below, and outputs a warning about text pairs _{P l} and forming a distance _{D l.} This warning indicates that there is a similar sentence pair in the news manuscript. Each pair of sentences may belong to the same news item, or may belong to different news items.

The sentence pair P _l detected by the pre-processing unit (811 or 911), the operator of the operation, or on the basis of the automatic operation or the like by the machine, one of these sentences or both statements, statements before and after By connecting with, similarity can be lowered. That is, by such connection, a sufficiently different difference (distance) can be taken between sentences detected as a pair. A good example will be described below as a method for connecting such sentences.

Sentence l (tilde) _x and sentence l (tilde) _y are both elements of the set M (tilde), and P _l = (l (tilde) _x , l (tilde) _y ) explain. That is, the sentence l (tilde) _x and the sentence l (tilde) _y form the minimum distance in the set M (tilde). There are four methods for concatenating sentences for either sentence l (tilde) _x or sentence l (tilde) _y .
(1) l (tilde) _x-1 and l (tilde) _x are connected to form a new sentence. Assume that the set of sentences after concatenation is M (tilde) ₀ .
(2) l (tilde) _x and l (tilde) _{x + 1} are connected to form a new sentence. Assume that the set of sentences after concatenation is M (tilde) ₁ .
(3) l (tilde) _y-1 and l (tilde) _y are connected to form a new sentence. Assume that the set of sentences after concatenation is M (tilde) ₂ .
(4) l (tilde) _y and l (tilde) _{y + 1} are connected to form a new sentence. Assume that the set of sentences after concatenation is M (tilde) ₃ .
Note that l (tilde) _x−1 and l (tilde) _{x + 1} are a sentence immediately before and a sentence after sentence l (tilde) _{x in} the document, respectively.

Thus, the preprocessing unit (811 or 911) generates M (tilde) ₀ , M (tilde) ₁ , M (tilde) ₂ , and M (tilde) ₃ which are four sentence sets as candidates. The preprocessing unit (811 or 911) calculates D _l according to the equation (18) for each of these four candidate sentence sets. Then, the preprocessing unit (811 or 911) is in the M _(tilde) 0, M _(tilde) 1, M _(tilde) 2, M _{(tilde) 3,} to adopt a connection method which maximizes the _{D l} . When the order of items read out in broadcasting is changed, or when a document can be replaced in units of items, the preprocessing unit (811 or 911) connects the sentences across the items. The above operation is performed under the restriction not to be performed. In this case, in other words, the preprocessing unit (811 or 911) uses only concatenation of sentences that do not cross items as candidates.

When there are a plurality of sentence pairs that give a value of D _l, the preprocessing unit (811 or 911) performs the following processing.
That is, the preprocessing unit (811 or 911), for one of the plurality of text pairs (the pairs, and l _{(tilde) x} and l _{(tilde) y),} l _{(tilde) x} and l (Tilde) A set M (tilde) is composed only of items including _y . Then, the preprocessing unit (811 or 911) searches the M (tilde) for the optimum sentence connection. How to connect sentences is as described above.
Then, the preprocessing unit (811 or 911) repeats such an operation for each of the plurality of sentence pairs.

Preprocessing unit (811 or 911) is until the D _l for a set M (tilde) is not below the threshold, repeating the operation of statements linked described above. The pre-processing unit (811 or 911) outputs a proposal for correcting a document based on the result of sentence concatenation. Then, based on the operation from the operator, the preprocessing unit (811 or 911) corrects the document and eliminates the similar sentence.
As described above, the preprocessing unit (811 or 911) performs an operation of connecting sentences so that the minimum distance in the set after connection is larger than that before connection.

FIG. 18 is a schematic diagram illustrating an example of a sentence pair detected by the preprocessing unit (811 or 911) and an example of linking one of the sentences with another sentence. As shown in the figure, sentences 1 to 11 are included in one item. This item is an example of an item including a similar sentence. A pair of sentence 2 and sentence 3 is extracted by the preprocessing unit (811 or 911) as a sentence pair similar to each other according to the expressions (18) and (19). When speech recognition error correction processing is performed based on a manuscript including such a sentence pair, an actual correct sentence with a series of state transitions corresponding to sentence 2 and a series of state transitions corresponding to sentence 3 May not be recognized as maximum likelihood. Specifically, the sentence 2 and the sentence 3 share the same word sequence of “Kanto / kara /// representative / 8 / school /// first match /// match” from the beginning of the sentence. After detecting that sentence 2 and sentence 3 are similar, the preprocessing unit (811 or 911) sets a plurality of connection methods as candidates as described above. That is, the case where sentence 1 and sentence 2 are connected, the case where sentence 2 and sentence 3 are connected, and the case where sentence 3 and sentence 4 are connected. Preprocessing unit (811 or 911), for collection in the case of their respective M (tilde) to calculate the D _l, adopts the way of connecting the values of D _l is the highest, and outputs it as proposed. In the same figure, the proposal that is output is to connect the original sentence 1 and sentence 2 to a new sentence 1. Sentence 1 after the concatenation is: “A summer national high school baseball season that will open at Koshien Stadium on the 6th of this month, today, a combined lottery was held. The opponents of the first eight matches from Kanto were decided.” It is.

[3. Detection of short sentences by the preprocessing unit]
The third warning situation is that a short sentence (a sentence that is too short) exists in the document.
For this reason, the preprocessing unit (811 or 911) outputs warning information indicating that the sentence is too short when the number of words included in the sentence is smaller than a predetermined threshold. Specifically, it is as follows.

  The preprocessing unit (811 or 911) counts the number of words included in each sentence. When the smallest number of words in each sentence included in the set M falls below a predetermined threshold, a warning regarding the sentence is output. At the same time, the preprocessing unit (811 or 911) outputs a proposal to perform an operation of linking the sentence with the preceding or succeeding sentence. For the connection with the previous sentence or the subsequent sentence, the one in which the smallest number of words among the sentences in the connected set M is larger is adopted. Also here, as described above, the pre-processing unit (811 or 911) does not perform sentence concatenation across items when the order of the items may be changed or the items themselves may be replaced. To. The pre-processing unit (811 or 911) performs necessary corrections in accordance with instructions from the operator.

  The pre-processing unit (811 or 911) repeats the short sentence detection and correction processes described above until the sentence with the smallest number of words in the set M has the number of words equal to or greater than the threshold. This can prevent errors due to the existence of sentences that are too short.

[4. Detection of similar sentence chunks by the preprocessing unit]
The fourth warning situation is that similar sentence chunks exist in the manuscript.
For this reason, the pre-processing unit (811 or 911) outputs warning information indicating that the beginning chunks are similar when the distance between the beginning chunks located at the beginning of the sentence is below a predetermined threshold. , Output a suggestion of modification that extends backwards at least one of the similar beginning chunks. Specifically, it is as follows.

When a sentence is handled by being divided into chunks, even if the first chunks of sentences are similar between sentences, it may be a factor to apologize for the correction of the recognition result. Here, for the first chunk set of each sentence included in the set M, the preprocessing unit (811 or 911) performs the following operations.
The set of the first chunk of each sentence is expressed by the following equation (20).

Then, the preprocessing unit (811 or 911) obtains D _c and P _c according to the following expressions (21) and (22).

D _c is the minimum value of the distance between the first chunks of each sentence. P _c is a pair of first chunks that forms such D _c . When D _c falls below a predetermined threshold (for example, 0.5), the preprocessing unit (811 or 911) outputs a warning regarding the chunk pair P _c .
In addition, the preprocessing unit (811 or 911) creates a plan for correcting the target chunk pair that issued the warning above. The process for that will be described next.

Regarding the chunks x and y included in the chunk pair P _c , x ′ and y ′ are chunks obtained by extending x and y backward by one word, respectively. Then, the preprocessing unit (811 or 911) obtains the following equation (23) using these x ′ and y ′.

  Then, the preprocessing unit (811 or 911) employs either x ′ or y ′ so as to realize the value of Expression (23). That is, x ′ is adopted when L (x ′, y)> L (x, y ′), and y ′ is adopted when L (x ′, y) <L (x, y ′). If L (x ′, y) = L (x, y ′), the shorter chunk (the smaller number of words) is extended. The preprocessing unit (811 or 911) outputs a proposal for eliminating a similar chunk at the beginning of a sentence. Then, the preprocessing unit (811 or 911) performs necessary corrections in accordance with an instruction from the operator.

Then, the preprocessing unit (811 or 911) repeats the above operation until the predetermined condition is satisfied, and extends the chunk. The predetermined condition is, for example, is to become a situation in which D _c does not fall below the threshold. When the pre-processing unit (811 or 911) satisfies the condition, the pre-processing unit (811 or 911) outputs the extended chunk as a correction proposal. By such an operation, the preprocessing unit (811 or 911) cancels the similarity between the first chunks between sentences.

FIG. 19 is a schematic diagram illustrating an example of similar chunks detected by the preprocessing unit (811 or 911) and an example of proposing extension of the chunk. In the illustrated example, chunks 386 and 392 are similar (including the case where they are the same). These chunks belong to different sentences. In this example, the chunks 386 and 392 are the same word string, and “the tourism in Gunma Prefecture has been recognized as a subject of reputational damage by this intermediate guideline”. Then, the preprocessing unit (811 or 911) is gradually extended by one word by the method described above, the state when both the distance D _c no longer fall below the threshold, "extension of the proposed chunks" shown in FIG. It is. As described above, the preprocessing unit (811 or 911) extends the chunk so that the difference between the chunks at the beginning of the sentence is sufficiently generated, thereby preventing an error in the speech recognition error correction.

[5. Edit distance between series]
As described above, in the processing by the preprocessing unit (811 or 911), the edit distance L (x, y) is calculated. As this editing distance, an improved version of the Levenshtein distance is used. The improved version used in the present embodiment is characterized by the edit distance between sequences. Specifically, this improved version does not count insertions and deletions at the end of the sequence. This is because in the present embodiment, it is important how to reduce the degree of matching near the beginning of the sequence, and it is not so important to reduce the degree of matching in the latter half of the sequence.

  FIG. 20 is a pseudo code of a program showing a calculation procedure for calculating an improved version of the Levenshtein distance between series. In this pseudo code, the character string “//” to the end of the line is a comment of the program and does not affect the execution code. The function LefteinDistance () defined in the first line of the source code receives two string type parameters str1 and str2. The function LeaventainDistance () returns an integer value. The function value returned is the improved Levenshtein distance value. lenStr1 and lenStr2 are the lengths of character strings that are input parameters, respectively. Note that lenStr1 ≦ lenStr2. The fourth line of the code is a declaration of an integer type array d used as a work area. Regarding the size of d, the first dimension (row) is (lenStr1 + 1), and the second dimension (column) is (lenStr2 + 1). The sixth line of code is an integer variable declaration. The variables i1 and i2 are for counting up the character strings str1 and str2, respectively. The variable cost is for temporarily storing the cost depending on whether or not the characters to be compared match between the two character strings. If the characters match, the cost is 0. If they do not match, the cost is 1. The loop by the for sentence from the 7th line to the 8th line and the loop by the for sentence from the 9th line to the 10th line initialize the array d. Specifically, when the array d is viewed as a matrix starting from the 0th row and the 0th column, each element in the 0th row and each element in the 0th column are initialized. Starting from the 11th line, the double nested for statement fills the table (array d) for calculating the Levenshtein distance while comparing the characters that are the elements of the character strings str1 and str2. It will be. The if sentence on the 13th line sets the cost between the positions to 0 if the corresponding characters are the same, and sets the cost to 1 if they do not match. The minimum () function on the right side of the assignment statement on the 15th line is a function that returns the minimum value among the parameters. With this assignment statement, when a character is inserted or deleted in comparison between both character strings, the edit distance is increased by 1 (counted up). Further, whether or not a character is replaced is represented by the above cost (variable cost), and the variable value is added. This array d represents the difference between character strings due to the minimum cost (editing operation). In other words, this array shows the path with the minimum energy for matching both character strings. The return statement in the last line returns an improved Levenshtein distance value. The difference between this program and the normal Levenstein distance calculation program is that it does not count insertions and deletions at the end of the sequence. The algorithm described here can be applied not only to character strings but also to word strings.

  FIG. 21 is a schematic diagram illustrating an execution example in which the improved version of the Levenshtein distance is calculated by the above program. In this execution example, a character series is processed, and there is seq1 = adcefg and seq2 = abcdefghij, and a distance between these two series is obtained. The execution example shown in the figure is a Viterbi node score matrix. That is, FIG. 21 shows the values stored in the array d when the processing of the function shown in FIG. In the code shown in FIG. 20, unnecessary parts of “thisrow” are removed and displayed. In the normal calculation of the Levenshtein distance, the lower right value (that is, “5”) of the matrix shown in FIG. 21 is calculated. In calculating the distance of the improved version used in the present embodiment, the minimum value (that is, “2”) in the lowest row of the illustrated matrix is calculated. This is because the insertion of the end of the sequence is ignored in the distance calculation of the improved version.

The first embodiment and the second embodiment described above may be modified as follows.
The pre-processing unit (811 or 911) does not necessarily have all of the four types of warning conditions described above (existence of duplicate items, existence of similar sentences, existence of short sentences (too short sentences), existence of similar chunks at the beginning of sentences). There is no need to deal with. The preprocessing unit (811 or 911) may correspond to only a part of these.
In addition, a word does not necessarily have to be a language unit. Instead, characters and phonemes may be used as language units, and similar processing may be performed based on the distance between sequences of such language units.
In the above, several threshold values (threshold value for distance between series, etc.) are mentioned. These threshold values may all be the same value, or may be different values.

  In addition, you may make it implement | achieve all or one part function of each apparatus in each embodiment mentioned above with a computer. In that case, a program for realizing these functions may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, a “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included, and a program that holds a program for a certain time. The program may be a program for realizing a part of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can be used to create content such as subtitles using a speech recognition result. The present invention can be used in, for example, a broadcasting business and other content providing businesses.

100 voice recognition error correction device 110 WFST storage means (corresponding document set storage means, finite state transducer information storage section)
120 Node data update means 130 Node data storage means 140 Document search means 141 Maximum score node detection means 142 Traceback means 143 Document division means 144 Output candidate storage means 145 Edit distance calculation means 146 Edit distance determination means 147 Final output storage means 148 Confirm Time storage means 150 Document output means 200 Document text set (document text storage section)
220 Speech recognition device 240 Transducer construction device 241 Word network registration means 242 Editing network registration means 600 Speech recognition error correction device 610 WFST storage unit (finite state transducer information storage unit)
620 Node data update unit 630 Node data storage unit 640 Document search unit 641 Maximum likelihood node detection unit 642 Traceback unit 643 Document division unit 644 Output candidate storage unit 645 Error score calculation unit 646 Error score determination unit 647 Confirmation output storage unit 648 Determination Time storage unit 650 Output unit 700 Document text storage device (document text storage unit)
720 Speech recognition device 740 Transducer construction device 741 Word network registration unit 742 Editing network registration unit 800,900 Speech recognition error correction device 811, 911 Preprocessing unit

Claims (5)

An original text storage unit for storing the original text;
With respect to a sequence of language elements included in the document text stored in the document text storage unit, a distance representing a similarity between the plurality of sequences is calculated, and whether the calculated distance is less than a predetermined threshold value If a similar sequence is detected, warning information is output when a similar sequence is detected, and the similarity is resolved by manipulating the original text for at least one of the detected similar sequences. And a preprocessing unit for correcting the manuscript text in accordance with an externally input instruction;
A finite state transducer that sequentially accepts words input as speech recognition results corresponding to the manuscript text, and performs state transition while accepting errors in words included in the speech recognition results Finite state transducer information storage that stores information about the state of the finite state transducer that makes a transition, and information including a transition source state, a transition destination state, an input symbol, an output symbol, and a transition weight regarding the state transition. And
A node data storage unit for storing a score representing the likelihood of the state in the finite state transducer;
While receiving an input of a recognition word that is a speech recognition result corresponding to the document text from outside, according to the received recognition word, a score at the time for each state in the finite state transducer is calculated, A node data update unit that updates the node data storage unit using the calculated score;
When an activation signal indicating the start of processing is received from the outside, the node data storage unit is referred to determine the maximum likelihood node at that time and refer to the finite state transducer information storage unit and the node data storage unit Then, by tracing back the state transition to the maximum likelihood node, a traceback process is performed up to a predetermined time when the state transition has been confirmed, and the path of the state transition subjected to the traceback process is set as an output candidate and output. With respect to the candidate path, an error score is calculated according to a ratio in which the state transition related to the error is included in the path, and the error score is smaller than a predetermined threshold based on the calculated error score. A manuscript search unit that uses a path output candidate as a final output;
An output unit for outputting the determined output obtained by the document search unit;
A speech recognition error correction apparatus comprising: The manuscript text is composed of a plurality of items,
The pre-processing unit outputs the warning information indicating that the items are duplicated when the distance between the items is below a predetermined threshold.
The speech recognition error correction apparatus according to claim 1. The manuscript text includes a plurality of sentences,
The pre-processing unit outputs the warning information indicating that the sentences are similar when the distance between the sentences is less than a predetermined threshold, and selects any of the similar sentences. Outputs a suggestion of modification to be linked to the sentence before or after the sentence,
The speech recognition error correction apparatus according to claim 1 or 2, The manuscript text includes a plurality of sentences,
The sentence is composed of one or more chunks,
The pre-processing unit outputs the warning information indicating that the head chunk is similar when the distance between the head chunks located at the head of the sentence is below a predetermined threshold, and is similar. Outputting a suggestion of modification to extend backwards at least one of the beginning chunks;
The speech recognition error correction apparatus according to any one of claims 1 to 3. The manuscript text includes a plurality of sentences,
The pre-processing unit outputs the warning information indicating that the sentence is too short when the number of words included in the sentence is smaller than a predetermined threshold;
The speech recognition error correction apparatus according to any one of claims 1 to 4, wherein

Images