JP4878220B2 - Model learning method, information extraction method, model learning device, information extraction device, model learning program, information extraction program, and recording medium recording these programs - Google Patents

Description

  The present invention relates to a technique for extracting data information.

  Conventionally, when extracting linguistic information from voice data or image data (hereinafter referred to as “original data”), for example, information such as a specific keyword, a person name, a place name, etc., using a technique such as voice recognition or character recognition, A word string or a character string estimated to be included in the data (hereinafter referred to as “recognized word string”) is obtained, and then information is extracted. In that case, the recognition word string may contain an error in speech recognition or character recognition (hereinafter referred to as “recognition error”), so that information that should be extracted is missing or an error that is not included in the original data. Information may be extracted. Therefore, it is necessary to reduce the influence of recognition errors.

  Information extraction technology using speech recognition uses a value called recognition confidence (confidence, hereinafter referred to as “confidence” as appropriate), which measures whether each recognition word is correct for the recognition result. There is a method for predicting a recognition error and reducing the influence of information extraction (see Non-Patent Document 1, for example).

  In this technique, a label (output label) indicating the type of an extraction target is assigned to a recognized word string using a generative model similar to a hidden Markov model that is a known technique. In this generation model, N that maximizes the joint probability P (W, N) of the word sequence W and the output label sequence N is selected. In general, since there is insufficient learning data to estimate the joint probability of the entire sequence, the joint probability between the word wi existing at a certain position i and the output label ni corresponding to the word wi is the immediately preceding I word and the immediately preceding word. An approximation is made that depends only on J output labels.

  The learning of the generation model in Non-Patent Document 1 uses data in which a recognition error is included as a word that includes a speech recognition error, and text data that does not include a recognition error. Each word of each data is given a label indicating the type to be extracted. And at the time of information extraction, using the recognition certainty of each recognized word, considering the possibility of recognition of the word and various possibilities such as the label to be given, the joint probability is finally optimal Output a series of labels. In this generation model, a word that is a recognition error is given a label so as not to be extracted. Therefore, when the generation model is used, for a recognition word with a low recognition certainty, the likelihood when the recognition word is processed as correct and the likelihood when the recognition word is processed as incorrect are considered. The effect of recognition error is reduced by selecting the recognition word after comparing with the originality.

  On the other hand, as a technique related to information extraction from normal character string data that does not consider recognition errors, a technique for extracting a specific expression in natural language processing is known. The proper expression is a proper noun such as a person name, a place name, and an organization name, and a noun phrase indicating a specific entity such as a date, time, and money. In the natural language processing, the identification expression (discriminative model) such as the maximum entropy model, support vector machine (hereinafter referred to as “SVM”) is used rather than the generation model such as the hidden Markov model. Is known to exhibit higher performance.

In this identification model, N that maximizes the conditional probability P (N | W) of the sequence N for the sequence W, not the simultaneous probability P (N, W) of the sequence W of words and the sequence N of output labels. select. The identification model is a model for determining which type (person name, place name, date, etc.) the identification object belongs to by using a number of features of the identification object. There is an advantage that information of part of speech etc. can be used flexibly. For example, Non-Patent Document 2 uses SVM to realize high-precision specific expression extraction by using words, parts of speech, and word character types (words consisting only of kanji, numbers,...) As features. Yes.
That is, the identification model can be used in an integrated manner with respect to a large number of pieces of information whose handled features are not independent of each other.

On the other hand, in the generation model, a combination of “word surface layer only” or “word surface layer + part of speech” is used as one feature in order to maintain the independence of the features. In this case, the number of features increases due to the combination of features, and the number of features increases exponentially as more information is integrated into the combined features, so the problem of data sparseness (learning data shortage) becomes larger. Become.
Therefore, in the generation model, using a large amount of information as is done in the identification model, that is, using the feature as information of the generation model has a large limitation on the information that can be used. Even if possible, it was difficult to realize.

ここで、Ｐ（Ｘ）をＸの確率、Ｐ（Ｘ，Ｙ）をＸとＹの同時確率、Ｐ（Ｘ｜Ｙ）をＸの条件付確率とする。 In addition, in the generation model, it is considered that “word surface layer” and “part of speech” are independent and simultaneously used as features separately, which causes an unnatural state.
This will be specifically described. Independence is represented by the following formula (1).

Here, let P (X) be the probability of X, P (X, Y) be the joint probability of X and Y, and P (X | Y) be the conditional probability of X.

For example, given a word (surface layer “ni”, part of speech “particle / case particle”), both are independent, ie, P (“ni” | “particle / case particle”) = P (“ni”) The conditional probability of observing “ni” as a word that is “particle / case particle” is equal to the probability of observing “ni” as an arbitrary word. From this, it can be seen that it is unnatural considering the frequency with which the part-of-speech of “ni” is “particle / case particle” (compared to the verbal form of the verb “niru”, the combined form, etc.).
Similarly, the word surface layer (for example, “NTT” (registered trademark)) and the character type (for example, “all alphabetic capital letters”) are not independent.
In addition, when using a hierarchical part-of-speech system and using multiple hierarchies as features, there is also a major part-of-speech classification (eg, “particle”) and a slightly finer classification (eg, “particle / case particle”). Just as independent.
Furthermore, methods that are not independent of each other can be considered as a method of giving character type features. For example, the word “THINK” can be considered as a word having three character type features of “alphabet”, “starting with an uppercase alphabetic character”, and “all uppercase alphabetic characters”.

Generating models that require independence of features requires the difficult task of defining an appropriate feature set that can maintain independence in order to use non-independent features such as the identification model. It will be.
Therefore, many identification models are used in the natural language processing field.

  Non-Patent Document 3 describes a technique for applying a maximum entropy model, which is an identification model, to a speech recognition result.

Patent Document 1 discloses a classification technique that significantly reduces the calculation time required for determining a class to which an input vector belongs in an identification model.
DDPalmer and M. Ostendorf, “Improving information extraction by modeling errors in speech recognizer output”, in Proceedings of the First International Conference on Human Language Technology Research, 2001 Hideki Amagasaki, Hideto Kazawa, “Acceleration of SVM for Extracting Specific Expressions”, Transactions of Information Processing Society of Japan, 2003, Vol.44, No.3, pp.970-979 L. Zhai et al., “Using N-best lists for named entity recognition from Chinese speech”, in Proceedings of Human Language Technology Conference of the North American Chapter of the Association of Computation Linguistics, 2004, pp. 37-40 JP 2003-331254 A

  Although the identification model is known to have high performance, the input word confidence (correct / predictive) information has not been used as in the generation model so far. However, there is only a technique of using an identification model that does not consider the presence of errors (see Non-Patent Document 3).

  In addition, when using correct / incorrect information of input words in the generation model as in Non-Patent Document 1, using the information of misrecognized word features (word itself, part of speech, character type, etc.) It was difficult from the viewpoint of feature independence, and could only be expressed as “a misrecognized word”, and flexible feature design was difficult.

The certainty factor in the generation model is a continuous value.
In order to learn model information using more information in an identification model, when a reference word string that does not include a recognition error and a recognition word string that includes a recognition error are used as model learning data, the reference word The same design features must be defined for sequences and recognition word sequences. Then, when using certainty factors such as those used in the generation model in order to give correct / incorrect information, the certainty factor of the reference word string is expressed as a binary value of “correct” or “false” Since the certainty factor of the recognized word string is a continuous value, it cannot be compared with the reference word string. That is, it is difficult to use information including confidence in learning of the identification model.

  Therefore, an object of the present invention is to solve the above-described problems and reduce the influence of recognition errors.

  In the present invention, the learning of the identification model does not include the recognition word string including the error to which the specific expression label (extraction target information) is added and the error to which the specific expression label (extraction target information) is similarly added. A reference word string (for example, a transcription of speech manually) is used. Then, by binarizing the recognition certainty feature indicating the correctness of each recognized word, the reference word string can be handled as “a recognized word string in which all recognized words are correct”. Thereby, the learning effect of the identification model can be improved by using the reference word string at the same time as the recognized word string.

  In the present invention, a feature related to a misrecognized word that was not used in the prior art is also used. Not only the information of “words that were misrecognized” as in the prior art using generative models, but also information about which words were recognized and what features the words have. In the present invention, it can be easily used. Information regarding such misrecognized words is information that has not been considered in the prior art using an identification model that assumes an input that does not include errors.

Therefore, in order to solve the above-mentioned problem, the model learning method according to claim 1, wherein model information for giving a label indicating a word type to each word included in input speech or character data is provided. Model learning data, which is speech or character data, a word sequence recognized by speech recognition or character recognition is used as a recognition word sequence, and the model learning data is used as a model learning method. A model learning device that learns the model information when a word string provided with a correct label indicating the type of the word is used as a reference word string for each word in the word string that is a corresponding correct recognition result , Information indicating that the recognition result is correct for each word in the recognition word string that matches the reference word string by comparing each word in the recognition word string and each word in the reference word string. Correct / incorrect information provided with recognition certainty by giving information indicating that the recognition result is incorrect to the words in the recognized word string that do not match the reference word string as recognition certainty A word string associating step for generating a false recognition word string, comparing the recognition word string with correct / incorrect information with the reference word string, and for each word in the recognized word string with correct / incorrect information that matches the reference word string A recognition word string learning data creation step of creating recognition word string learning data having a recognition certainty factor and a label by giving a label given to the reference word string that matches the word; and the reference word each word in the column, the recognition result that imparts information indicating that correct as the recognition confidence, the reference word sequence learning to create a reference word sequence learning data and a labeled recognition certainty factor And over data producing step, as an input the recognized word sequence training data and the reference word sequence training data, support vector machines, maximum entropy models or by using conditional random field, at least the recognition certainty as features, the A model creation step of learning the model information for assigning a label indicating an optimum word type to each word included in the input speech or character data, and storing the model information in a storage unit. And

  According to such a procedure, the model learning device can compare the recognized word sequence and the reference word sequence to which the correct / incorrect information is given, thereby creating the model information, so that the accuracy at the time of information extraction is improved.

An information extraction method according to claim 2 is an information extraction method for extracting a word with a label indicating a predetermined type as extraction target information from voice or character input data . The model learning data, which is data, is a recognition word string that is a word string recognized by speech recognition or character recognition, and each word in the word string is a correct recognition result corresponding to the model learning data. When a word string with a correct label indicating the word type is used as a reference word string, a label indicating the word type is assigned to each word included in the input speech or character data. A model learning device that learns model information for performing comparison between each word in the recognized word string and each word in the reference word string, and the recognized word string that matches the reference word string Information indicating that the recognition result is correct is given as a recognition certainty, and information indicating that the recognition result is incorrect for a word in the recognition word string that does not match the reference word string is used as the recognition certainty. A word string associating step for generating a recognized word string with correct / incorrect information having a recognition certainty, and comparing the recognized word string with correct / incorrect information with the reference word string to match the reference word string The recognition word string learning data including the recognition certainty factor and the label is created by giving a label given to the reference word string that matches the word to each word in the recognition word string with the correct / incorrect information. A recognition word string learning data creation step, and providing each word in the reference word string with information indicating that the recognition result is correct as the recognition certainty factor, thereby providing a recognition certainty factor and a label. Using a support vector machine, a maximum entropy model, or a conditional random field, with reference word string learning data creating step for creating reference word string learning data, and input of the recognized word string learning data and the reference word string learning data The model information for giving a label indicating the optimum word type to each word included in the speech or character input data is learned and stored in the storage means with at least the recognition certainty as a feature. A model creation step; a word string recognition step of recognizing the input data of the voice or characters into a word string by voice recognition or character recognition; and creating a word lattice representing a plurality of recognized word string candidates in a graph expression ; For each word included in the word lattice, a score representing the correctness of recognition of the word as a continuous value If the score is equal to or higher than a predetermined threshold, information indicating that the recognition is correct is given as a recognition certainty, and otherwise, information indicating that the recognition is incorrect is used as the recognition certainty for each word. by imparting to, using the word confidence calculating step of creating a recognized word sequence with confidence information for the word lattice having a recognition confidence, the model information created in the modeling step, for the word lattice for each word in confidence information with recognized word sequence, wherein the labeling step of applying a label, the certainty factor information-recognized word sequence for the word lattice, a word label corresponding to the predetermined type is granted characterized in that it comprises an information extraction step to extract as the extraction target information.

  According to this procedure, the information extraction device calculates the certainty factor of the recognized word string as correct / incorrect information and assigns it to the recognized word string. Thereby, information can be extracted using the extraction object information given based on correct / incorrect information.

The model learning device according to claim 3 is a model learning device that learns model information for assigning a label indicating a word type to each word included in input speech or character data. Corresponding to word sequence recognition means for creating a recognition word sequence that is a sequence of words recognized by speech recognition or character recognition of data for model learning that is speech or character data, and the data for model learning For each word in the word string that is the correct recognition result, a word string that is given a correct label indicating the type of the word is referred to as each word in the reference word string that is a reference word string and each word in the recognized word string Is added to the word in the recognized word string that matches the reference word string as information indicating that the recognition result is correct, and the recognition unit that does not match the reference word string. Recognition result of a word in the string to confer information indicating that an error as a recognition confidence, the word string correlating means for generating errata with recognized word sequence having a recognition confidence, the errata The word recognition word string is compared with the reference word string, and a label given to the reference word string that matches the word is assigned to each word in the recognition word string with correct / incorrect information that matches the reference word string Recognition word string learning data creation means for creating recognition word string learning data having a recognition certainty factor and a label, and information indicating that the recognition result is correct for each word in the reference word string by imparting a recognition confidence, the reference word sequence learning data generating means for generating a reference word sequence learning data and a recognition confidence and labels, the recognition word sequence training data and the reference word sequence learning de As input data, support vector machine, maximum entropy models or by using conditional random field, at least the recognition certainty as features, optimal for each word contained in the data of the input voice or character It comprises a model creation means for learning the model information for giving a label indicating the type of word and storing it in a storage means.

  According to such a configuration, the model learning device can compare the recognized word sequence to which the correct / incorrect information is given and the reference word sequence, thereby creating the model information, thereby improving the accuracy at the time of information extraction.

Further, the information extraction device according to claim 4, the input data of voice or text, an information extracting device for extracting the extraction target information words label indicating the predetermined type is applied, the speech during model training Alternatively, a recognition word string that is a string of words recognized by voice recognition or character recognition is created from model learning data that is character data, and the voice or character input data is voice-recognized or character-recognized at the time of information extraction. And a word string recognition means for creating a word lattice representing a plurality of recognized word string candidates in a graph expression, and a word that is a correct recognition result corresponding to the model learning data during model learning A word string in which a correct label indicating the type of the word is assigned to each word in the string is referred to as each word in the reference word string that is a reference word string and the recognition word. Each word in the word string is compared, and information indicating that the recognition result is correct is given to the word in the recognized word string that matches the reference word string as a recognition certainty, and does not match the reference word string A word string associating means for generating a recognition word string with correct / incorrect information having a recognition certainty by giving information indicating that a recognition result is an error to a word in the recognition word string as a recognition certainty; The recognition word string with correct / incorrect information is compared with the reference word string during model learning, and each reference word string in the recognition word string with correct / incorrect information that matches the reference word string is added to the reference word string that matches the word. Recognizing each word in the reference word string at the time of model learning by recognizing word string learning data creating means for creating recognition word string learning data having a recognition certainty factor and a label by giving a given label result By giving information indicating correctness as the recognition certainty, reference word string learning data creating means for creating reference word string learning data having a recognition certainty factor and a label, and the recognized word string learning at the time of model learning Data and the reference word string learning data are input and included in the input speech or character data using at least the recognition confidence as a feature using a support vector machine, a maximum entropy model, or a conditional random field. Learning model information for giving a label indicating the optimum word type for each word, storing the model information in the storage means, and for each word included in the word lattice at the time of information extraction , Calculate a score representing the correctness of recognition as a continuous value, and if the score is equal to or greater than a predetermined threshold, the recognition is correct Is given as a recognition certainty factor, and in other cases, information indicating that the recognition is incorrect is given to each word as a recognition certainty factor , so that the certainty factor for the word lattice having the recognition certainty factor a word confidence measure calculation means for creating information-recognized word sequence, using the model information created by said model creating means, against the time information extraction each word confidence information with recognized word sequence for the word lattice, the label a labeling means for applying, from confidence information-recognized word sequence for the word lattice during information extraction, and information extracting means to extract a word label corresponding to said predetermined type is assigned as the extraction target information It is characterized by providing.

  According to this configuration, the information extraction device calculates the certainty factor of the recognized word string as correct / incorrect information and assigns it to the recognized word string. Thereby, information can be extracted using the extraction object information given based on correct / incorrect information.

Further, the model learning program according to claim 5, the model learning process according to claim 1, characterized in that to be executed by a computer.
According to such a configuration, the computer in which the model learning program is installed can realize each function based on this program.

An information extraction program according to claim 6 causes a computer to execute the information extraction method according to claim 2 .
According to this configuration, the computer in which the information extraction program is installed can realize each function based on this program.

The storage medium according to claim 7, characterized in that it is a model learning programs or computer-readable recording medium which records information extraction program according to claim 6, of claim 5. Examples of the recording medium include a hard disk, a CD-ROM, a DVD, a flexible disk, and a memory. By reading this storage medium into a computer, each function of the information extraction program can be executed on an arbitrary computer.

  According to the present invention, the influence of recognition errors can be reduced.

Hereinafter, the best mode (hereinafter referred to as “embodiment”) for carrying out the model learning method and the information extraction method of the present invention will be described in detail with reference to the drawings.
In the present embodiment, model learning is to create (update) model information used at the time of information extraction, and information extraction is information extraction using input object information for input data. Is.

[Configuration of terminal device]
FIG. 1 is a functional block diagram showing an example of a terminal device (model learning device, information extraction device) according to the present invention.
As illustrated in FIG. 1, the terminal device 100 includes an input / output unit 110, a control unit 130, and a storage unit 150.

The input / output means 110 is composed of, for example, an input / output interface, etc., and acquires input data and commands, or outputs predetermined data.
Here, the input / output unit 110 acquires the input data, acquires the data input unit 111 that outputs the data to the control unit 130, and acquires the extraction result information transmitted from the control unit 130, and outputs the information to a display, a printer, or the like. Data output means 115.

  The control means 130 is composed of, for example, a CPU (Central Processing Unit) and the like, and controls the input / output means 110 and the storage means 150 and extracts information of the input data, as shown in FIG. , A recognition unit 131, a learning data processing unit 133, a model creation unit 135, and an information extraction unit 137.

The recognition unit 131 acquires the input original data from the data input unit 111 and outputs a recognition word string and the like. The original data is model learning speech data, character data, data to be extracted, and the like.
The recognition unit 131 includes a word string recognition unit 1310 that performs voice recognition or character recognition, and a word certainty factor calculation unit 1311 that calculates the certainty factor of each recognized word.

The word string recognition means 1310 outputs one or more recognized word strings for the input original data (learning input data) during model learning. Further, at the time of information extraction, a word lattice (a directed acyclic graph for compactly expressing a plurality of recognized word strings) is created for the input original data and output to the word certainty calculation means 1311. .
As an example of these voice recognition techniques, for example, Reference 1 is cited.
[Reference 1]
Lee Sung-nobu, Kawahara Tatsuya, Doshita Shuji, “Large Vocabulary Continuous Speech Recognition Parser Based on A * Search Using Grammar Category Pair Constraints”, IPSJ Journal, 1999, Vol.40, No.4, pp.1374 -1382
A specific explanation of the word lattice will be given later.

The word certainty calculating means 1311 calculates the certainty (recognition certainty) of each recognized word using information (word lattice) output from the word string recognizing means 1310 at the time of information extraction. As a specific calculation method, a word certainty calculation technique known in the speech recognition field can be used. For example, in the method described in Reference 2, the word posterior probability is calculated as a ratio of “the sum of the scores of speech recognition hypotheses including the target word” and “the total score of all speech recognition hypotheses obtained from the word graph”.
[Reference 2]
F. Wessel et al., “Confidence measures for large vocabulary continuous speech recognition”, IEEE transaction on Speech and Audio Processing, 2001, vol9, No.3, pp.288-298

The word certainty calculating means 1311 calculates the recognition certainty and creates a recognized word string with certainty. The recognition certainty (correct answer / incorrect answer score) of the calculated recognition certainty is a continuous value. If the value is large, the recognition of the word is likely to be correct, and if the value is small, the possibility of an error is high. Therefore, the word certainty calculating means 1311 creates a recognized word string with certainty information to which a value of “correct” is given if the recognition certainty is equal to or greater than the threshold, and “false” otherwise.
In addition, by including the word certainty calculation means 1311 in the word string recognition means 1310, it can be executed simultaneously with the recognition of the word string. In addition, the word string recognition unit 1310 can also execute a score such as a likelihood corresponding to the recognition certainty degree as an input by giving the recognition word string a score.

The learning data processing unit 133 receives a recognition word string for the input original data and a reference word string to which extraction target information (for example, a specific expression label) corresponding to the original data is input, for learning an identification model Output data.
Here, the reference word string is a label (unique) corresponding to identification information indicating which type of extraction target information each word corresponds to a correct word string (manual transcription result, etc.) corresponding to the original data. Expression label). This reference word string and the original data input to the terminal device 100 are stored in a learning source database (hereinafter referred to as “learning source DB” as appropriate), and the learning source database is input to the terminal device 100. As a result, the model learning process is started.

  The learning data processing unit 133 includes a word string associating means 1330 for determining a recognition error in the recognized word string, a recognition word string learning data creating means 1331 for creating learning data using a recognition word string with correct / incorrect information, a reference word string And a reference word string learning data creating means 1332 for creating learning data using the.

  The word string associating unit 1330 associates the recognized word string with the reference word string (correct word string) by DP (Dynamic Programming) matching, which is a known technique, and associates each recognized word with the associated reference. If it is the same as the word, “correct”, otherwise “false”, a recognition word string with correct / incorrect information to which correct / incorrect information of the recognized word is added is output.

  The recognition word string learning data creation unit 1331 extracts features used in the identification model for the recognition word string with correct / incorrect information created by the word string association unit 1330. Furthermore, the recognition word string with correct / incorrect information and the specific expression label of the reference word string in the learning source database are compared and matched to each other, and a label indicating which extraction target information corresponds to each recognition word (specific expression label) Is stored in the model learning database 151 as recognized word string learning data 1510.

  The feature design excluding the recognition certainty is not particularly defined in the present embodiment, but examples include “word”, “part of speech”, “character type”, and the like described in Non-Patent Document 2.

FIG. 2 is a diagram illustrating an example of recognized word string learning data stored with the recognition certainty feature added. Note that the label (extraction target information) given here is based on the following.
-------
BEGIN: Label given to the first word of the proper expression (person name-BEGIN, etc.)
MIDDLE: Labels given to intermediate words in proper expressions (person names-MIDDLE, etc.)
END: Label given to the last word of the proper expression (person name-END, etc.)
SINGLE: A label attached to a unique expression consisting of one word (person name-SINGLE, etc.)
-------
In FIG. 2, it can be seen from the label information that “place name” and “artifact name” are designated as extraction target information.

In the present embodiment, the terminal device 100 assigns the label attached to the corresponding reference word as it is when the recognized word is correct, and other labels (for example, extraction) when the recognition word is incorrect. A label “not applicable (OTHER)” indicating that information to be included is not included.
When the information to be extracted extends over a plurality of words, if even one word included in the information is a recognition error, another type of label (excluded) is similarly given to the other words included therein. It is also possible. For example, in FIG. 2, “Kyoto” and “Kinkakuji” that are correctly recognized are assigned “place name” and “artifact name” as label information, respectively. Here, if the information to be extracted is “Omiya Station” consisting of the two words “Omiya” and “Station”, “Omiya Station” is “bound to Omiya” due to misrecognition. BEGIN's "Omiya" is recognized (recognition certainty feature = "correct"), but "ge name-END""station" is misrecognized (recognition certainty feature = "false") Including “Omiya” of “Place name-BEGIN”, it is not subject to extraction (that is, the label information is “Not applicable”).

Returning to FIG. 1, the reference word string learning data creation unit 1332 extracts features used in the identification model in the reference word string, and further assigns recognition confidence features (assuming all recognition words are correct) to the reference word string. Further, a specific expression label is assigned and stored as reference word string learning data 1511 in the model learning DB 151.
FIG. 3 is an example of reference word string learning data. The contents correspond to those in FIG.

The model creation unit 135 includes a model creation unit 1350. The model creation unit 1350 receives the learning data (reference numerals 1510 and 1511) of the identification model and outputs the created model information.
Specifically, the model creation means 1350 reads the learning data (reference numerals 1510 and 1511) from the model learning database 151 and learns so that a correct label sequence can be assigned to the word string in the input data at the time of information extraction. I do. Although the type of the identification model and its learning method are not particularly defined in the present embodiment, SVM (support vector machine), maximum entropy model, conditional random field (Conditional Random Fields), and the like can be considered. Note that Non-Patent Document 2 can be cited as a technique using SVM.

The information extraction unit 137 inputs a recognized word string and certainty factor for the original data, and extracts information from the recognized word string by using learned model information.
The information extraction unit 137 uses the model information 1520 created by the model creation unit 135 to label the recognition word string, and provides information extraction corresponding to the specified extraction target information. Information extraction means 1371 to be provided.

The label giving means (extraction target information giving means) 1370 receives the recognition word string with certainty information from the word certainty degree calculating means 1311 and uses the model information 1520 to identify which label each recognition word corresponds to. And an optimal label is assigned based on the calculated result. Regarding the use of the identification model, a technique according to the type of model such as SVM, maximum entropy model, conditional random field, or the like is used.
FIG. 4 is a diagram showing an example of a result of label assignment by the label attaching unit 1370 when “America's presidential palace is the White House” is input as an example of original data at the time of information extraction.

  The information extraction unit 1371 performs information extraction with reference to the label provided by the label addition unit 1370. For example, in the labeling result illustrated in FIG. 4, if the specified extraction target information is “artifact name”, the word “white house” is extracted.

  The storage unit 150 includes, for example, a RAM (Random Access Memory) and an HDD (Hard Disk Drive). In this case, the RAM is used for arithmetic processing by the control unit 130 and stores information acquired via the input / output unit 110. The HDD stores various databases, predetermined programs, and processing of the control unit 130. Stores results etc.

  Further, the storage unit 150 includes a model learning database (hereinafter referred to as “model learning DB” as appropriate) 151 created by the learning data processing unit 133 and model information created by the model creating unit 135. A database (hereinafter referred to as “model information DB” as appropriate) 152 is provided.

  The model learning DB 151 includes recognition word string learning data 1510 and reference word string learning data 1511. The recognition word string learning data 1510 stores learning data created from the recognition word string. The reference word string learning data 1511 stores learning data created from the reference word string.

  The model information DB 152 stores information such as classes, features, and parameters as model information 1520.

  Each of the means 1310 to 1371 included in the control means 130 is realized by the CPU developing and executing a predetermined program stored in the HDD of the storage means 150 on the RAM.

[Operation of information extraction device]
The operation of the terminal device 100 shown in FIG. 1 will be described with reference to FIGS. 5 and 14 (refer to FIG. 1 as appropriate). FIG. 5 is a flowchart showing a model learning method performed by the terminal device shown in FIG. FIG. 14 is a flowchart illustrating an information extraction method performed by the terminal device illustrated in FIG. Each process of model learning and information extraction will be described by taking, as an example, extraction of specific expressions from speech data read from newspaper articles.

<Model learning>
The learning source database (learning source DB) used in this embodiment includes newspaper article data (reference word string) that has been subjected to morphological analysis, and voice data that reads out the newspaper article (original data: one article into one voice file). Included). The learning source database corresponds to the learning input data in the claims.

The reference word string is stored in the learning source DB in the format illustrated in FIG. 6, and the first column is word information corresponding to the correct word string (surface layer ten reading + part of speech information). Indicates the information on the specific expression label as an example of the extraction target information.
The specific expression label is information indicating the characteristics (attributes) of each word used at the time of information extraction. By referring to this label, the word is extracted at the time of information extraction.

In the present embodiment, eight types of specific expression labels of person name, place name, organization name, artifact name, date, time, amount, and ratio are given as examples of the specific expression label. A word that does not belong to any of them is labeled “OTHER”. In addition, in order to represent a specific expression over a plurality of words, in addition to the eight types of specific expression labels, a label including information on the position of the following words is used (except “OTHER”).
-------
BEGIN: Label given to the first word of the proper expression (person name-BEGIN, etc.)
MIDDLE: Labels given to intermediate words in proper expressions (person names-MIDDLE, etc.)
END: Label given to the last word of the proper expression (person name-END, etc.)
SINGLE: A label attached to a unique expression consisting of one word (person name-SINGLE, etc.)
-------
Accordingly, in this embodiment, there are 33 types of labels of 8 types × 4 positions + OTHER.

  The terminal device 100 acquires learning input data via the data input unit 111 and stores it in the storage unit 150 as a learning source DB. Then, the recognition unit 131 acquires original data (voice data) and a reference word string included in the learning source DB.

  The word string recognition unit 1310 receives the original data (voice data) as input and performs recognition using a known voice recognition technique (S1: word string recognition step). The recognized result is obtained in the form of a recognized word string as shown in FIG.

  In the recognition word string illustrated in FIG. 7, the reference numeral 701 indicates the word string of the first recognition candidate, and the reference numeral 702 indicates the word string score (“-30922.7”) and its breakdown (acoustic model score (“ AM = -35053.9 ") and the language model score (" LM = 4131.28 ")).

Subsequently, the word string associating unit 1330 associates the reference word string (correct word string) of the learning source DB with the recognition word string obtained in step S1, and based on this, the recognition word string with correct / incorrect information is obtained. Create (S2: word string association step).
Specifically, first, the word string associating unit 1330 compares the recognized word string with the reference word string (correct word string) and performs association. FIG. 8 is a diagram illustrating the association result. The first column is the reference word sequence (reference word side), the second column is the recognition word sequence (recognition word side), and there is no corresponding word (where the word was inserted / dropped during recognition) (null) Is marked.
The word string associating unit 1330 uses the information of the word string associating result shown in FIG. 8 and is “correct” when the word on the reference word side coincides with the word on the recognized word side. The recognition word string with correct / incorrect information shown in FIG. 9 is created by adding “false” information to the recognized word string for the thing (that is, (null) or mismatch).

  Subsequently, the recognized word string learning data creating unit 1331 uses a label (specific expression label) indicating the extraction target information given to the reference word string and the recognized word string with correct / incorrect information, such as DP matching which is a known technique. The recognition word string learning data shown in FIG. 10 is created by using the method (S3: recognition word string learning data creation step).

The following four are used as examples of word features used in the present embodiment.
-------
Word surface features Part-of-speech features Character type features (Hiragana, Katakana, single kanji, kanji, etc.)
Recognition certainty feature (correction information of recognition word obtained from matching result in learning data)
-------

  That is, the recognized word string learning data creating unit 1331 creates recognized word string learning data by adding these features and the unique expression label to the recognized word string as a result of the correspondence.

  In the recognition word string learning data shown in FIG. 10, the first column is a word surface feature, the second column is a part of speech feature, the third column is a character type feature, the fourth column is a recognition certainty feature, and the fifth column is a specific expression label. It is.

  In the present embodiment, when the unique expression includes a recognition error in the recognized word string, all of the recognized words corresponding to the specific expression are assigned labels other than the unique expression (OTHER).

  That is, for example, the person name “Murayama Tomi City” in the input data is recognized as “Murayama” “Mr.” “Ni” “Position” “Sh” “Use” in the recognition word string as shown in FIG. It is causing an error. In this “Murayama Tomi City”, “person name-BEGIN” to “person name-END” indicate person names in the unique expression label of the reference word string (see FIG. 6). In FIG. 10, “Tomiichi” caused a recognition error and was replaced with another word, so “Murayama” is “positive” in the recognition confidence feature, but the corresponding “person name-BEGIN” The unique expression label is not attached, and “OTHER” is assigned.

The recognized word string learning data creating unit 1331 stores the created recognized word string learned data in the model learning DB 151 of the storage unit 150 as recognized word string learned data 1510.
Note that the information in the recognition word string learning data of the model learning database shown in FIG. 5 and the information included in the recognition word string learning data shown in FIG. 10 are: recognition word string = word surface layer feature, feature = (part of speech feature, (Character type feature), correct / incorrect information = recognition certainty feature, extraction target information = specific expression label, and so on.

  The reference word string learning data creating unit 1332 creates reference word string learning data to which the same features and recognition certainty factors as the recognition word string learning data 1510 are assigned to the reference word strings (S4: reference word string). Learning data creation step). At that time, since all the words are considered to be correct recognition words in the reference word string, the recognition certainty feature is “positive” for all the words. FIG. 11 is a diagram illustrating an example of reference word string learning data created by the reference word string learning data creation unit 1332.

The reference word string learning data creating unit 1332 stores the created reference word string learning data as reference word string learning data 1511 in the model learning DB 151 of the storage unit 150.
Note that the information in the reference word string learning data of the model learning database shown in FIG. 5 and the information included in the reference word string learning data shown in FIG. 11 are: reference word string = word surface layer feature, feature = (part of speech feature, (Character type feature), correct / incorrect information = recognition certainty feature, extraction target information = specific expression label, and so on.

  Subsequently, the model creation unit 1350 of the model creation unit 135 creates model information using the recognized word string learning data 1510 and the reference word string learning data 1511 (S5: model creation step).

  In the present embodiment, SVM, which is a well-known technique, is used as an identification model for extracting a specific expression. As implementation of SVM, TinySVM published as free software is used, and YamCha published as free software is used to realize the model creation means 1350.

  Since SVM is a binary classifier, it cannot be directly applied to a multi-class classification problem such as eigenexpression extraction. Therefore, an SVM that classifies whether or not it belongs to each class is created for each class, and the classification to multiple classes is based on the result of all those SVMs, and the class showing the highest score is the correct one-against- A method called all (or one-versus-rest) was used. YamCha automatically creates learning data for each SVM by one-against-all from the learning data, learns SVM by TinySVM, and finally summarizes the parameters of SVM (33 in this embodiment). The model information 1520 is stored in one file. In the present embodiment, in addition to the features (4 types) of the words to be labeled, the features of 2 words before and after (4 types × 4 words = 16 types) are simultaneously used. The YamCha option corresponds to “F: -2..2: 0 ..” and “MULTI_CLASS = 2”.

  The SVM kernel for extracting the specific expression is a second-order polynomial kernel, and the soft margin is set to “0.1”. It corresponds to “-d 2 -C 0.1” in the TinySVM option.

As a result, model information (class / feature description portion) is written as model information 1520 as illustrated in FIG.
Here, first, information that is commonly used in all SVMs is written as shown in FIG.
In ClassList :, a list of labels given by the model information for information extraction is output.
Subsequently, feature number definitions are output. For example, in the feature No. 18, its own word surface layer (“F” is a feature, “+0” is a word position, “0” is a column position in learning data) is “AP communication”. It is shown that.
For example, in the reference word string learning data shown in FIG. 11, a feature for giving a specific expression label to the first word (word surface feature “Murayama”) will be described. The next word (word surface feature “rich” When the part-of-speech feature “noun-proper noun” of “city” is represented by the model information of FIG. 12, it is indicated as “F: +1: 1: noun-proper noun”. Similarly, when the character type feature “kanji” of the next word (word surface feature “prime”) is expressed by model information, “F: +2: 2: kanji” is indicated.

Subsequently, as illustrated in FIG. 13, model information 1520 includes SVM parameters (support vectors and their weights, etc.) used in each SVM to determine whether they belong to each class in the learning data. Added to and exported.
In FIG. 13, reference numeral 1301 indicates a part of SVM parameters for determining whether a certain word is classified into the label “artifact name-BEGIN”. Reference numeral 1302 indicates that each row represents a support vector. First, the support vector weight is output, and then a list of feature numbers and feature values of the support vector is output.

The model information 1520 is created by the above processing. When new learning source data is input, the model information 1520 stored in the model information DB 152 is updated by performing the processes of steps S1 to S5. Specifically, when the parameter portion of the model information shown in FIG. 13 is updated and a new label is added or a new word is added, the information of FIG. 12 is also updated. .
That is, model information including correct / incorrect information and label information can be created, and the accuracy in extracting information is improved. The results of this experiment will be described later.

<Information extraction>
Here, the information extraction process will be described with reference to FIG. It is assumed that the model information 1520 is stored in the terminal device 100 in advance in steps S1 to S5 in FIG. As the original data (input data) for extracting information, “the United States occupied Hong Kong in the Opium War” is used.

First, word string recognition unit 1310, using the original data obtained through the input output means 110 (input data), to create a word lattice (S11: word sequence recognition step).

  FIG. 15 is a diagram illustrating an example of a word lattice created by the word string recognition unit 1310. The word lattice in the present embodiment takes the form of a weighted finite state transducer (WFST), and each line is represented as a state transition. The first column is the transition source state number, the second column is the transition destination state number, the third to fourth columns are the time frames in the voice corresponding to this state transition (the start time of the audio file is “0”, the time “1”) Corresponds to 20 ms), the fifth column is an input symbol (here, “eps”: no input symbol), the sixth column is an output symbol (recognition word), and the seventh column is a state transition weight (score).

Subsequently, the word certainty calculation means 1311 performs certainty calculation using the word lattice, and creates a recognized word string with certainty information (S12: word certainty calculation step).
In this embodiment, the certainty factor calculation method using SVM is used as a method for calculating the certainty factor of the word here. However, the method is not limited to this method as long as a correct / incorrect answer label can be assigned. Absent.
In addition, as features for calculating the word certainty factor, the following is used as an example.
-------
Word surface layer Part of speech number Word posterior probability (divided into 10 levels: (1) greater than 0 and less than 0.1, (2) greater than 0.1 and less than 0.2, ..., (10) greater than 0.9 and less than 1.0)
-------

  In the present embodiment, the features for the two words before and after are also used at the same time as when creating a model for extracting a specific expression. The calculation of the word posterior probability uses the posterior probability calculation method using the word graph described in Reference Document 2 described above. And, for the implementation of SVM for calculating word confidence using information such as word posterior probabilities, use SVM that uses YamCha and TinySVM to determine whether each recognized word is correct (CORRECT) or error (ERROR). Learning (creation) is performed using the word certainty factor calculation SVM learning data illustrated in FIG. Note that another method may be used to calculate the word posterior probability and the word certainty factor. In that case, the SVM learning data for word certainty calculation illustrated in FIG. 16 may be unnecessary.

Here, an example of processing in which the recognized word string learning data creation unit 1331 generates the word certainty calculation SVM learning data illustrated in FIG. 16 will be described.
During model learning, the word string recognition unit 1310 creates a word lattice using the original data.
Then, the recognition word string learning data creation means 1331 uses the word lattice received from the word string recognition means 1310 and the recognition word string with correct / incorrect information (see FIG. 9) to calculate word confidence SVM learning data (see FIG. 9). 16). The “correct / incorrect label” in the SVM learning data for word certainty calculation (see FIG. 16) is given in correspondence with the correct / incorrect information in the recognized word string with correct / incorrect information (see FIG. 9).

  Learning with SVM learning data for word certainty calculation is a binary classification problem, so there is no need to use one-against-all, and one SVM that determines whether or not CORRECT (correct answer) is learned. That's fine. In this embodiment, the SVM kernel for calculating word certainty is a second-order polynomial kernel, and the soft margin is set to “0.01”. In TinySVM options, this is equivalent to "-d 2 -C 0.01".

  The SVM parameters learned using the word certainty calculation SVM learning data are stored in the model information DB 152 in the same manner as the processing of the model creation means 1350 during model learning.

  As described above, the process of generating the SVM learning data for calculating word certainty (see FIG. 16) and storing it in the model information DB 152 is performed by using SVM in step S12. This method can be omitted.

When a word lattice is input from the word string recognition unit 1310 at the time of information extraction, the word certainty factor calculation unit 1311 creates recognition word string data for word certainty factor calculation with a word a posteriori probability feature for the word certainty factor calculation. To do. FIG. 17 is a diagram illustrating an example of recognition word string data for word certainty calculation. When this recognition word string data for word certainty calculation (“that” is misrecognition of “opium”) is input to YamCha, model information is created from the SVM learning data for word certainty calculation and stored SVM parameters. Is used to obtain the output shown in the word certainty calculation result of FIG. The columns added from the data in FIG. 17 are the score determined to be CORRECT and the score determined to be ERROR (the sign inversion of the score in the case of “CORRECT”), and the certainty of the extraction target information It corresponds to. Subsequently, in this embodiment, the score c (s) obtained by normalizing the output score s of the SVM to a value in the range of 0 to 1 using the sigmoid function shown in Expression (2) is used as the confidence of the recognized word. Use as a degree.

  FIG. 19 shows values obtained by normalizing the score of “CORRECT” of “Score of correct / incorrect answer” in FIG. 18 using Expression (2).

In the present embodiment, “0.4” is used as the threshold value used in determining the certainty factor feature, and the normalized value is determined. If the threshold value is equal to or greater than the threshold value, information “correct” is given and information “error” is given otherwise. And added as a recognition certainty feature. The result is shown in FIG. 20 as a recognized word string with certainty information. Through the above procedure, the word certainty factor calculation unit 1311 creates data in which a feature including the recognition certainty factor is added to the recognized word string composed of the word lattice.
Through the above processing, correct / incorrect information is added to the recognized word string of the input data.

Subsequently, the label providing unit 1370 performs labeling on the recognized word string with certainty factor information (see FIG. 20) input from the word certainty factor calculating unit 1311 (S13: extraction target information adding step).
Specifically, a recognition word string with certainty factor information (see FIG. 20) is input from the word certainty factor calculating unit 1311 to the label providing unit 1370. Then, the label assigning means 1370 uses the model information 1520 created by the model creating means 1350 (step S5 in FIG. 5) to label the input recognition word string with certainty information using, for example, SVM. I do. The result is output as shown in FIG. A part added in comparison with FIG. 20 is a label and a score of the label (output score of each SVM). Although the scores for all the output labels are output, in the example of FIG. 21, the output labels after the third candidate are omitted.

  If the output of FIG. 21 is used as a labeling result as it is, an inconsistency may occur as a label string (in the case of “USA”, “artificial name-BEGIN” comes after “place name-BEGIN”, etc.). In the following manner, an optimum label sequence satisfying the label concatenation constraint is obtained.

  First, the label score s is normalized to a value in the range of 0 to 1 using a sigmoid function (see equation (2)). Thereafter, a label sequence that maximizes the sum of normalized scores c (s) is selected by a known Viterbi algorithm or the like (see Non-Patent Document 2). By this procedure, the output of the above-mentioned YamCha is corrected, and the result as illustrated in FIG. 22 is obtained.

Subsequently, the information extraction unit 1371 extracts a portion (“BEGIN” to “END” or “SINGLE”) corresponding to the unique expression from the output of the label assigning unit 1370 (S14: information extraction step). The extracted information is output via the data output means 115 as appropriate.
For example, in the case of the data illustrated in FIG. 22, a specific expression label is given.
-------
United States Place name Hong Kong Place name
-------
Is output.

According to the present embodiment, in the identification model, each of the recognized word string learning data 1510 and the reference word string learning data 1511 has a binary feature (recognition certainty feature) indicating whether each recognized word is correct or not. Therefore, it is possible to create highly accurate model information and extract information using correct / incorrect information at the time of information extraction using the model information. Thereby, the influence of recognition errors can be reduced. That is, the word feature “that” misrecognized in the original data is not extracted as a specific expression by the output label (OTHER).
In addition, not only correctly recognized words but also misrecognized words, various feature information of the words can be used.

  The terminal device 100 can also be realized by executing a model learning program that causes a general computer to execute the above-described steps S1 to S5. The terminal device 100 can also be realized by executing an information extraction program that causes a general computer to execute the above-described steps S11 to S14. These programs can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

[Experiment]
The method and results of the experiment by the method shown in this embodiment are shown below.

<Experimental conditions>
・ Original data: Speech readings of newspaper articles (1,174 articles, 10,718 sentences, 106 readers (about 100 sentences per person))
-Reference word string: Document data of the newspaper article (letters that cannot be read such as parentheses are deleted, and all numbers are replaced with Chinese numerals)
-Information to be extracted: Specific expressions (artifact names, organization names, place names, person names, dates, timetables, monetary values, ratios) in the reference word string, total approximately 19,000

The document data of the reference word string is obtained by performing morphological analysis using a well-known Japanese morphological analyzer “tea bowl” and giving word break and part of speech information.
The word character types are “one Kanji character” (one Kanji character), “Kanji” (only one or more Kanji characters), “Hiragana” (Hiragana only), “Katakana” (only Katakana), “numbers” (numbers only), “ "Uppercase one letter" (English uppercase one letter), "Uppercase" (English uppercase only), "Uppercase start" (Only the first letter is uppercase), "English" (other words only in English letters), "Other" (Other ).

The speech recognizer used for creating the recognition word string used an acoustic model created with another reading speech and a word trigram model created from another newspaper article data. The language model weight for speech recognition was set to “15”.
Under this condition, the word recognition accuracy for reading speech of 10,718 sentences was 79.45%, and 82.0% specific expressions remained in the speech recognition result.

<Evaluation index>
As an evaluation index, an F value of proper expression, which is generally used in the field, was used. The F value is a harmonic average of the precision and the recall, and the precision and the recall are represented by the following formulas (3) and (4), respectively.

  The extracted proper expression is correct when speech recognition is correctly performed, there are no word errors, there are no words in excess or shortage, and the proper expression label (person name, place name, etc.) is correct. Was an error.

<Compared method>
(1) No certainty information: A method of extracting a specific expression from only text data (reference word string) without using certainty information (corresponding to Non-Patent Document 2). Use speech recognition results as text.
(2) Comparative example (A): Utilization of confidence information. Text data (reference word string) is not used in learning data.
(3) Comparative Example (B): Utilization of certainty information. In the learning / tesquite data, when the recognition certainty feature is “false”, the word feature / part-of-speech feature / character type feature is not used (the information “misrecognized word” is used).
(4) This embodiment: Uses certainty factor information. All functions of this embodiment are used.
(5) Upper limit value of the present embodiment: The method according to the present embodiment when it is assumed that the correctness of the recognized word is obtained without error. Upper limit of performance obtained in this embodiment.

<Experimental result>
The experimental results are shown in FIG.
Compared with the case where the certainty factor information is not used (there is no certainty factor information), the present embodiment shows a performance as high as 2.0% in terms of the F value. This performance improvement is due to a 7.4% improvement in precision (reproduction rate is reduced by 1.9%).

  Although the comparative example (A) is inferior to the present embodiment, the F value can be improved by 0.7% by using the certainty factor information. As a reason why the comparative example (A) is inferior to the present embodiment, the text data (reference word string) is not used for learning the model in the comparative example (A). This is thought to be due to the lack of learning data in places that are lost due to errors. That is, the effect of this embodiment by introducing the recognition certainty factor into the identification model is that the reference word string data is also used for learning the model as the recognition word string data with no error (and the recognition certainty factor is continuously used for that purpose). It was confirmed that further improvement could be obtained by introducing a framework of binarization instead of value.

  The comparative example (B) is almost similar to the present embodiment, but is slightly inferior to the present embodiment (0.3% in F value and 0.6% in conformity). The reason for this is that this embodiment uses more information (features related to misrecognized words) than the comparative example (B), and the use of such features contributes to the performance improvement of this embodiment. It was recognized that Such a feature has been difficult to use in a conventional method for extracting a specific expression using a generated model. Therefore, the effect can be obtained only by using this embodiment.

  In the present embodiment, by using the recognition certainty information, it is possible to perform specific expression extraction in consideration of words that are likely to be erroneously recognized. For this reason, it is possible to prevent misrecognized words from being extracted by including them in the proper expression, which is considered to have led to an improvement in the precision. Further, the result of the “upper limit value of the present embodiment” suggests that further improvement in performance can be expected in the information extraction field by improving the accuracy of recognition confidence.

  As described above, according to the present embodiment, by using the identification model, the recognition error is converted into correct and incorrect binary feature information, and the reference word string is compared with the recognized word string. By performing learning, it is possible to improve accuracy during information extraction. In addition, features related to misrecognized words can also be used.

As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. For example, in this embodiment, word features, label types, and contents can be freely set.
In addition, the technique using the known technique is not limited to this, and there is no problem even if it is carried out using another method.
Also, in the applicable technical field, the present invention can be applied in various fields other than the information search system, the natural language processing system, and the speech processing system.

It is a functional block diagram which shows an example of the terminal device (model learning apparatus, information extraction apparatus) which concerns on this invention. It is a figure which shows the example of recognition word sequence learning data. It is a figure which shows the example of reference word sequence learning data. It is a figure which shows the result to which the label was provided. It is a flowchart which shows the model learning method which the terminal device shown in FIG. 1 performs. It is a figure which shows the example of the reference word string stored in the learning origin database. It is a figure which shows the example of the recognition word sequence produced from original data. It is a figure which shows the result as which word string matching means matched. It is a figure which shows the example of the recognition word sequence with correct / incorrect information. It is a figure which shows the example of recognition word sequence learning data. It is a figure which shows the example of reference word sequence learning data. It is a figure which shows the example of model information (class and feature description part). It is a figure which shows the example of model information (parameter description part). It is a flowchart which shows the information extraction method which the terminal device shown in FIG. 1 performs. It is a figure which shows the word lattice created from the input data at the time of information extraction. It is a figure which shows the example of SVM learning data for word reliability calculation. It is a figure which shows the example of the recognition word sequence data for word certainty calculation at the time of information extraction. It is a figure which shows the example of the word reliability calculation result at the time of information extraction. It is a figure which shows the data which normalized the score in FIG. It is a figure which shows the example of the recognition word sequence with certainty factor information. It is a figure which shows the example of a label provision result. It is a figure which shows the corrected label provision result. It is a figure which shows an experimental result.

Explanation of symbols

100 terminal device (model learning device, information extraction device)
DESCRIPTION OF SYMBOLS 110 Input / output means 111 Data input means 115 Data output means 130 Control means 131 Recognition part 133 Learning data processing part 135 Model creation part 137 Information extraction part 150 Storage means 151 Model learning database 152 Model information database 1310 Word string recognition means 1311 Word Certainty factor calculation means 1330 Word string association means 1331 Recognition word string learning data creation means 1332 Reference word string learning data creation means 1350 Model creation means 1370 Label assignment means (extraction information addition means)
1371 Information extraction means 1510 Recognition word string learning data 1511 Reference word string learning data 1520 Model information

Claims (7)

A model learning method for learning model information for providing a label indicating a word type for each word included in input speech or character data ,
The model learning data, which is speech or character data, is a recognition word string that is a word string recognized by speech recognition or character recognition, and is a correct recognition result corresponding to the model learning data. When a word string to which a correct label indicating the type of the word is assigned to each word is used as a reference word string,
A model learning device for learning the model information,
Recognize certainty information by comparing each word in the recognized word string with each word in the reference word string and indicating that the recognition result is correct for the word in the recognized word string that matches the reference word string The recognition word with correct / incorrect information having the recognition certainty is given to the word in the recognition word string that does not match the reference word string as the recognition certainty by giving information indicating that the recognition result is incorrect. A word sequence matching step for generating a sequence;
The recognized word string with correct / incorrect information is compared with the reference word string, and each word in the recognized word string with correct / incorrect information that matches the reference word string is given to the reference word string that matches the word A recognition word string learning data creation step for creating recognition word string learning data having a recognition certainty factor and a label by giving a label ;
Reference word string learning for creating reference word string learning data having a recognition certainty factor and a label by giving information indicating that a recognition result is correct to each word in the reference word string as the recognition certainty factor A data creation step;
Using the recognition word string learning data and the reference word string learning data as input, using a support vector machine, a maximum entropy model, or a conditional random field , at least the recognition confidence as a feature, the input speech or character And a model creation step of learning the model information for assigning a label indicating the optimum word type to each word included in the data and storing the model information in a storage means. An information extraction method for extracting, as extraction target information, a word provided with a label indicating a predetermined type from speech or character input data,
The model learning data, which is speech or character data, is a recognition word string that is a word string recognized by speech recognition or character recognition, and is a correct recognition result corresponding to the model learning data. When a word string to which a correct label indicating the type of the word is assigned to each word is used as a reference word string,
A model learning device that learns model information for assigning a label indicating the type of a word to each word included in the input speech or character data,
Recognize certainty information by comparing each word in the recognized word string with each word in the reference word string and indicating that the recognition result is correct for the word in the recognized word string that matches the reference word string The recognition word with correct / incorrect information having the recognition certainty is given to the word in the recognition word string that does not match the reference word string as the recognition certainty by giving information indicating that the recognition result is incorrect. A word sequence matching step for generating a sequence;
The recognized word string with correct / incorrect information is compared with the reference word string, and each word in the recognized word string with correct / incorrect information that matches the reference word string is given to the reference word string that matches the word A recognition word string learning data creation step for creating recognition word string learning data having a recognition certainty factor and a label by giving a label;
Reference word string learning for creating reference word string learning data having a recognition certainty factor and a label by giving information indicating that a recognition result is correct to each word in the reference word string as the recognition certainty factor A data creation step;
Using the recognition word string learning data and the reference word string learning data as inputs, using a support vector machine, a maximum entropy model, or a conditional random field, at least the recognition confidence as a feature, and input data of the speech or characters A model creation step of learning the model information for assigning a label indicating an optimum word type for each word included in the storage, and storing the model information in a storage unit;
A word string recognition step of recognizing the input data of the voice or characters into a word string by voice recognition or character recognition, and creating a word lattice representing a plurality of recognized word string candidates in a graph expression ;
For each word included in the word lattice, a score expressing the correctness of recognition of the word as a continuous value is calculated, and information indicating that the recognition is correct if the score is equal to or greater than a predetermined threshold value as the recognition certainty The recognition word string with certainty information for the word lattice having the recognition certainty is created by adding to each word information indicating that the recognition is incorrect otherwise as the certainty of recognition. A word confidence calculation step;
Using the model information created in the modeling step, For each word in confidence information with recognized word sequence for the word lattice, and labeling step of applying a label,
Wherein the confidence factor information with the recognized word sequence for the word lattice, the information extraction method characterized by including the information extracting step to extract a word label corresponding to said predetermined type is assigned as the extraction target information. A model learning device that learns model information for assigning a label indicating a word type to each word included in input speech or character data ,
Word string recognition means for creating a recognition word string that is a string of words recognized by voice recognition or character recognition from data for model learning that is voice or character data ;
Each word in the reference word string that is a reference word string is a word string in which a correct label indicating the type of the word is assigned to each word in the word string that is a correct recognition result corresponding to the data for model learning Is compared with each word in the recognized word string, and information indicating that the recognition result is correct is given to the word in the recognized word string that matches the reference word string as a recognition certainty, and the reference word string A word string association that generates a recognition word string with correct / incorrect information having a recognition certainty by giving information indicating that the recognition result is incorrect to a word in the recognized word string that does not match with the recognition certainty Means,
The recognized word string with correct / incorrect information is compared with the reference word string, and each word in the recognized word string with correct / incorrect information that matches the reference word string is given to the reference word string that matches the word A recognition word string learning data creating means for creating recognition word string learning data having a recognition certainty factor and a label by giving a label ;
Reference word string learning for creating reference word string learning data having a recognition certainty factor and a label by giving information indicating that a recognition result is correct to each word in the reference word string as the recognition certainty factor Data creation means;
Using the recognition word string learning data and the reference word string learning data as input, using a support vector machine, a maximum entropy model, or a conditional random field , at least the recognition confidence as a feature, the input speech or character A model learning apparatus comprising: a model creating unit that learns the model information for assigning a label indicating an optimum word type to each word included in the data and stores the model information in a storage unit. An information extraction device that extracts, as extraction target information, a word to which a label indicating a predetermined type is given from voice or character input data,
Creates a recognition word sequence that is a sequence of words recognized by speech recognition or character recognition from the data for model learning that is speech or character data during model learning, and the input data of the speech or characters during information extraction A word string recognition means for recognizing a word string by recognition or character recognition and creating a word lattice representing a plurality of recognized word string candidates in a graph expression ;
In a reference word string that is a reference word string, a word string in which a correct label indicating the type of the word is assigned to each word in the word string that is a correct recognition result corresponding to the model learning data at the time of model learning Each word of the recognition word string and each word of the recognition word string, the word in the recognition word string that matches the reference word string, the information indicating that the recognition result is correct is given as a recognition certainty, A word that generates a recognition word string with correct / incorrect information having a recognition certainty by giving information indicating that the recognition result is incorrect to a word in the recognized word string that does not match the reference word string as a recognition certainty. Column association means;
The recognition word string with correct / incorrect information is compared with the reference word string during model learning, and each reference word string in the recognition word string with correct / incorrect information that matches the reference word string is added to the reference word string that matches the word. A recognition word string learning data creating means for creating recognition word string learning data having a recognition certainty factor and a label by giving the given label;
Reference for creating reference word string learning data having a recognition certainty factor and a label by giving information indicating that a recognition result is correct to each word in the reference word string at the time of model learning as the recognition certainty factor Word string learning data creation means;
The recognition word string learning data and the reference word string learning data are input at the time of model learning, and the input is performed using at least the recognition certainty as a feature using a support vector machine, a maximum entropy model, or a conditional random field. Model creation means for learning model information for assigning a label indicating an optimum word type for each word included in speech or character data, and storing it in a storage means;
For each word included in the word lattice at the time of information extraction, a score expressing the correctness of recognition of the word as a continuous value is calculated, and information indicating that the recognition is correct is recognized if the score is equal to or greater than a predetermined threshold. Recognized word string with certainty information for the word lattice provided with the certainty of recognition by giving information indicating that the recognition is incorrect otherwise to each word as the certainty of recognition. A word certainty calculation means for creating
Using the model information created by said model creating means, For each word in confidence information with recognized word sequence for the word lattice during information extraction, and the label applying means for applying the labels,
From confidence information-recognized word sequence for the word lattice during information extraction, information, comprising an information extracting means to extract a word label corresponding to said predetermined type is assigned as the extraction target information Extraction device. Model learning program for the model learning process according to claim 1, causes the computer to execute. An information extraction program for causing a computer to execute the information extraction method according to claim 2 . A computer-readable recording medium in which the model learning program according to claim 5 or the information extraction program according to claim 6 is recorded.

Images