JP6062879B2 - Model learning apparatus, method and program - Google Patents

Description

  The present invention relates to a model learning apparatus, method, and program, and more particularly, to a model learning apparatus, method, and program for extracting a specific expression to which a label indicating a category of specific expression is assigned.

  Traditionally, from the sentence “The Nile is the longest river in the world”, an expression that points to the unique substance “Nile River”, “proprietary expression” is automatically extracted, and a detailed label (for example, “river name”) There is known a technique for imparting (for example, Non-Patent Document 1). A proper expression is an expression that refers to a specific place or thing, such as a proper noun. For example, “NTT (registered trademark)” and “Osaka” are specific expressions for “organization” and “location”. Here, “organization” and “location” are called categories of specific expressions. Conventionally, eight named entity categories defined in a conference called IREX have been used as standard named entity categories. However, “location” and “organization” are coarse in granularity, and more detailed categorization may be necessary depending on the application. Sekine et al. Of New York University have proposed the concept of extended named expressions that subdivided named expressions, and 200 named classes are defined in that framework (Non-patent Document 2).

  Further, in Non-Patent Document 1, the extended specific expression is solved by conventional clue information (peripheral context information) and CRF which is one of machine learning methods. Here, CRF (Conditional Random Fields) is a discriminator / learner used to solve a sequence labeling problem such as morphological analysis and extraction of specific expressions.

Also, there is a method for extracting a specific expression using a “learning data” in which a specific expression label is given to text by the identification learning device of Non-Patent Document 1 and creating a model for giving a detailed label. Known (for example, Patent Document 1).
Also, a technique for acquiring an extended specific expression dictionary from a specific expression and label pair using an existing dictionary such as Wikipedia is known (for example, Non-Patent Document 3).

JP 2013-246795

Hashimoto, Nakamura, “Constructing Corpus with Extended Specific Expression Tags—White Paper, Book, Yahoo! Wisdom Bag Core Data”, 16th Annual Conference of the Language Processing Society of Japan, March 2010 Satoshi Sekine, Chikashi Nobata, Definition, dictionaries and tagger for Extended Named Entity Hierarchy LREC2004 pp. 1977-1980 Ryuichiro Higashinaka, Kugatsu Sadamitsu, Kuniko Saito, Toshiro Makino and Yoshihiro Matsuo, "Creating an Extended Named Entity Dictionary from Wikipedia", The 24th International Conference on Computational Linguistics (COLING 2012)

  However, in the method of Patent Document 1, there is a problem that learning data must be created manually, and costs are increased when the accuracy of the model is increased.

  Further, since the unique expression is generated every day, matching using only the dictionary obtained by Non-Patent Document 3 has a problem in the reproduction rate of the specific expression.

  In the present invention, there is provided a model learning device, a method, and a program, which are made to solve the above-described problem, and are capable of learning a model for accurately labeling an extraction target character string. Objective.

  In order to achieve the above object, a model learning apparatus according to the present invention includes first learning data that is a document in which a label is pre-assigned to a character string to be extracted, and a document that is not provided with the label. Collating an input unit that receives input of certain second learning data, a character string included in a document of the second learning data, and a dictionary in which the character string and the label are stored in association with each other; A dictionary collation unit for assigning the label to a character string that matches the character string stored in the dictionary in association with the label in the second learning data document; A feature is extracted from the learning data to be a target domain feature, the target domain feature is duplicated to be a common domain feature, and includes the target domain feature, another domain feature, and the common domain feature. Generating a feature vector, extracting the feature from the second learning data in which a label is assigned to the character string to obtain the other domain feature, replicating the other domain feature to the common domain feature, and the feature vector Each of the target domain feature, the other domain feature, and the common domain feature based on the feature vector of the first learning data and the feature vector of the second learning data A learning unit that extracts a parameter for the common domain feature from the learned parameters, extracts a character string to be extracted, and outputs the extracted character string as a model parameter; , Including.

  In the model learning device according to the present invention, the character string to be extracted is a specific expression, the label is a specific expression classification, the model is extracted from the specific expression, and the specific expression is included in the specific expression. It is good also as a model for giving a classification.

  In the model learning method according to the present invention, the input unit is a first learning data that is a document in which a label is previously given to a character string to be extracted, and a second document that is a document to which the label is not given. A step of accepting input with learning data, and a dictionary collation unit collates a character string included in the document of the second learning data with a dictionary in which the character string and the label are stored in association with each other; In the document of the second learning data, a step of assigning the label to a character string that matches the character string stored in the dictionary in association with the label; Extracting a feature from first learning data to obtain a target domain feature, duplicating the target domain feature to obtain a common domain feature, the target domain feature, another domain feature, and the common domain Generating a feature vector consisting of sex, extracting the feature from the second learning data in which a label is attached to the character string to be the other domain feature, duplicating the other domain feature to be the common domain feature, Generating the feature vector; and a learning unit based on the feature vector of the first learning data and the feature vector of the second learning data, and the target domain feature, the other domain feature, and the As a parameter of a model for learning a parameter for each common domain feature, extracting a parameter for the common domain feature from the learned parameters, extracting a character string to be extracted, and assigning the label And an output step.

  Moreover, the program of this invention is a program for functioning a computer as each part which comprises said model learning apparatus.

  As described above, according to the model learning device, method, and program of the present invention, the dictionary is used to assign a label to the character string of the second learning data to which no label is assigned, and the first Extracting features from the learning data and the second learning data, generating a feature vector consisting of the target domain feature, the other domain feature, and the common domain feature for the first learning data and the second learning data, Learning the parameters for each of the target domain feature, other domain feature, and common domain feature, and extracting the parameters for the common domain feature to learn a model for accurately labeling the extraction target character string The effect of being able to be obtained is obtained.

It is a block diagram which shows the functional structure of the model learning apparatus which concerns on embodiment of this invention. It is a flowchart which shows the model learning process routine in the model learning apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the method of learning a model. It is a figure which shows the replication example of a target domain feature and other domain features.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Principle of Embodiment of the Present Invention>

  In the embodiment of the present invention, a model for extracting a character string and assigning an appropriate label is learned by using transfer learning and existing learning data and learning data obtained from WEB or the like.

  Transfer learning is a method of learning not only data of a target domain (for example, a newspaper) but also data of another domain (for example, a blog) so as to be effective for the target domain during model learning.

  The transfer learning method is described in Non-Patent Document 4 (Hal Daume III, “Frustratingly Easy Domain Adaptation”, In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 256-263 (ACL-2007)). The feature augmentation method described above can be used, and the same method is used in the embodiment of the present invention.

  The Feature Augmentation method uses the first learning data (existing learning data), which will be described later, as the target domain, and the second learning data (which is obtained by adding a pseudo correct label to a document set obtained from WEB etc.) As a domain, a target domain model is learned including other domains. In the Feature Augmentation method, learning is performed in consideration of domains by duplicating many features for each domain. Any learning method may be used.

  For example, it is assumed that features a and c having a weight of 1.0 are obtained from a document in the target domain. Further, it is assumed that features a, b, and d having a weight of 1.0 are obtained from documents in other domains. Features c and d are contradictory (for example, in the target domain, “// case particle” was always modified to “ga / case particle”, but in other domains it was always modified to “// case particle”. Such a case). The Feature Augmentation method takes the union of feature types and handles the features a, b, c, and d. Then, the features a, b, c, and d are copied to each of “common”, “target domain”, and “other domains” to construct a feature vector.

  At the time of learning, in the case of a feature obtained from the target domain, the features of the “common” and “target domain” are validated to generate a feature vector. In the case of features obtained from other domains, feature vectors of “common” and “other domains” are validated to generate feature vectors.

  When the weight of each feature is optimized using the feature value calculated in this way, the feature weight that appears in common in the target domain and other domains becomes larger in the “common” feature. A feature that appears only in one of the target domain and the other domain is also given a weight of the feature in “common”. On the other hand, for features that conflict in the target domain and other domains, the weight of the feature in the “target domain” or “other domain” increases, but the weight of the feature in “common” approaches zero.

  The embodiment of the present invention is characterized in that pseudo correct answer data is used for the second learning data which is another domain used for transfer learning. Specifically, by searching for a document with no label from WEB or the like, and using an existing dictionary (which stores a pair of character string and label), a pseudo learning data is obtained by assigning a label to the corresponding location. obtain. For example, if a pair of a character string and a label “Tanaka-person name” is in the dictionary, learning data “Today / Tanaka (pseudo correct answer = person name) / has met” is obtained from an unlabeled document.

  However, this pseudo learning data has the following problems.

  First, there is a problem that the word-label correspondence in the dictionary may be incorrect. For example, a “product name” label may be given to “Oda Nobunaga”.

  There is also a problem that ambiguity may exist in words. For example, since the word “Tanaka” includes “person name” and “place name”, the peripheral context that is a clue cannot be considered only by the dictionary, and the label to be given cannot be determined.

  In addition, there is a problem that the possibility of including other labels in the context cannot be considered. For example, “Today / Tanaka (pseudo correct answer = person name) / and // Yoshida (the correct answer is a person's name, but it is not in the dictionary, it will be NIL) /”. Can not.

  In the embodiment of the present invention, in order to solve these problems, data artificially created using transfer learning is regarded as another domain, and original learning data is simultaneously learned as a target domain.

  In the embodiment of the present invention, for the sake of easy understanding, specific expression extraction will be described as an example.

<Configuration of Model Learning Device According to Embodiment of the Present Invention>

  Next, the configuration of the model learning device according to the embodiment of the present invention will be described. As shown in FIG. 1, the model learning device 100 according to the embodiment of the present invention inputs a character string that has been given a label and has been subjected to morphological analysis as an input, Learn the model for extracting and labeling. This model learning apparatus 100 is composed of a computer including a CPU, a RAM, and a ROM that stores a program for executing a model learning processing routine described later, and is functionally configured as follows. Yes. As illustrated in FIG. 1, the model learning device 100 includes an input unit 10, a calculation unit 20, and an output unit 50.

  The input unit 10 receives a set of morphologically analyzed documents (teacher data) in which a correct label is assigned in advance to a character string to be extracted as a first set of learning data. Further, the input unit 10 accepts a set of morpheme-analyzed documents that are not given an explicit label, such as a WEB text, as a second set of learning data.

  The arithmetic unit 20 includes a teacher database 30, a document set database 32, a dictionary database 34, a dictionary collation unit 36, a transfer learning feature generation unit 38, a model learning unit 40, and a model storage unit 42. It consists of

  The teacher database 30 stores a set of first learning data.

  The document set database 32 stores a second set of learning data.

  The dictionary database 34 stores a dictionary that stores pairs of character strings and labels to be extracted. In the embodiment of the present invention, a manually created dictionary is used. Further, it is possible to use an extended specific expression dictionary that stores a pair of specific expression and specific expression classification label (Tonegawa-River pair, etc.) obtained by the method described in Non-Patent Document 5. It is also possible to create a dictionary storing a pair of a specific expression and a specific expression classification label by placing a virtual label for the specific expression from a dictionary having no label.

  The dictionary collating unit 36 collates the entry of the character string and label pair included in the dictionary stored in the dictionary database 34 with respect to each of the second learning data stored in the document set database 32 and performs dictionary lookup. I do. As a result of the dictionary lookup, if a character string that matches the character string stored in the dictionary in association with the specific expression classification label is included in the document of the second learning data, the matching character string is The document of the 2nd learning data to include is extracted, and a pseudo correct label is provided with respect to the applicable character string.

  In order to control the number of pseudo learning data, the following restrictions may be provided. For example, the number of matching cases may be limited to the total number of cases N. Further, the number of matching cases N_L for each label may be limited. Further, the number of matching cases N_E for each entry in the dictionary may be limited. For example, if N_L = 100, the label may be assigned to learning data of up to 100 cases for each label.

  The transfer learning feature generation unit 38 extracts, for each of the first learning data, the feature of each word included in the document of the first learning data. As a feature to be extracted, a feature similar to the conventional one is used (Patent Document 1). Taking the case of extracting a proper expression as an example, the document of the first learning data is “Niigata / O / Flow / <River: Shinanogawa> / Has / Japan / Long / River”. When the word is “Shinano River” and the unique expression classification label “river name” is given, label information such as “the word is [river name]”, “the last character in the word is“ river A feature indicating in-word information such as “is” and peripheral context information such as “the next word is“ ha ”” is generated. As a result, the featured text as shown in FIG. 3A is output. In addition, the second learning data document given the pseudo-correct answer label is “Kanto / no / <River: Tonegawa> / ha / awesome”, the word of interest is “Tonegawa”, and the proper expression classification In the case where the label “river name” is given, a featured text as shown in FIG. 3B is output. In the example of FIG. 3, a B tag is assigned to the word at the beginning of the extraction range of the specific expression, an I tag is assigned to a word other than the beginning of the extraction range, and an O tag is assigned to other words. . In the present embodiment, the sequential label features before and after the case where a plurality of unique expressions appear are not used.

  Then, the transfer learning feature generation unit 38 duplicates each feature extracted for each word of the first learning data as a target domain feature, and further obtains a common domain feature by duplicating the target domain feature. Specifically, for each extracted feature, prefixes such as IN_ and COMMON_ are added in front of an identifier representing the feature type and copied (see FIG. 4A). IN_ represents the target domain, and COMMON_ represents the common domain. The transfer learning feature generation unit 38 then duplicates the target domain feature, the other domain feature having a value of 0, and the common domain feature that duplicates the target domain feature for each word of the first learning data document. Generate a feature vector consisting of

  Similarly, the transfer learning feature generation unit 38 extracts the feature of each word included in the second learning data document for each of the second learning data. The transfer learning feature generation unit 38 duplicates each feature extracted for each word of the second learning data as another domain feature, and further obtains a common domain feature by duplicating the other domain feature. Specifically, for each extracted feature, prefixes such as OUT_ and COMMON_ are added in front of the identifier representing the feature type and copied (see FIG. 4B). OUT_ is for other domains. Then, the transfer learning feature generation unit 38, for each word of the second learning data document, the target domain feature with a value of 0, the duplicated other domain feature, and the common domain feature that duplicated the other domain feature Generate a feature vector consisting of

  In this way, the transfer learning feature generation unit 38 includes a feature vector generated for each word of the first learning data and a feature vector generated for each word of the second learning data. Is generated.

  The model learning unit 40 is based on a transfer learning feature file composed of a feature vector of each word of the first learning data generated by the transfer learning feature generation unit 38 and a feature vector of each word of the second learning data. Then, the machine learner learns parameters for each of the target domain feature, the other domain feature, and the common domain feature. Note that learning by the machine learning device may be performed by using a conventionally known method (for example, the method described in Non-Patent Document 1), and thus description thereof is omitted.

  Then, the model learning unit 40 extracts a parameter for the common domain feature from the parameters of the target domain feature, the other domain feature, and the common domain feature obtained by learning, and extracts a character string to be extracted. The parameter is stored in the model storage unit 42 as a model parameter for giving a label, and is output to the output unit 50.

<Operation of Model Learning Device According to Embodiment of Present Invention>

  Next, the operation of the model learning device 100 according to the embodiment of the present invention will be described. When the input unit 10 receives the first learning data set and the second learning data set, the model learning device 100 executes a model learning processing routine shown in FIG.

  First, in step S100, a dictionary stored in the dictionary database 34 is read.

  Next, in step S102, a set of first learning data stored in the teacher database 30 is acquired.

  In step S104, the second learning data to be processed is acquired from the second learning data set stored in the document set database 32.

  In step S106, the second learning data document acquired in step S104 is subjected to dictionary lookup using the dictionary read in step S100, and matches the character string stored in the dictionary in association with the label. If the character string to be included is included in the second learning data document, the second learning data document including the matching character string is extracted, and a pseudo-correct answer label is assigned to the corresponding character string. To do.

  In step S108, it is determined whether or not the processing in step S106 has been executed for all the second learning data. If there is second learning data that has not been executed in step S106, Returning to step S104, the second learning data is set as a processing target. On the other hand, when the process of step S106 is executed for all the second learning data, the process proceeds to step S110.

  In step S110, for each of the first learning data received by the input unit 10, each feature is extracted for each word included in the document of the first learning data.

  Next, in step S112, for each word in each document of the first learning data, each feature extracted in step S110 is duplicated to obtain a target domain feature and a common domain feature. A feature vector composed of another domain feature having a value of 0 and a common domain feature is generated.

  In step S114, each feature is extracted for each word included in the document of the second learning data with respect to each of the second learning data in which the pseudo correct answer label is assigned to the character string in step S106.

  Next, in step S116, for each word of each document of the second learning data, each feature extracted in step S114 is duplicated to obtain another domain feature and a common domain feature, and the value is set to 0. A feature vector including a domain feature, another domain feature, and a common domain feature is generated.

  In step S118, the machine learner uses the feature vector for each word of the first learning data generated in step S112 and the feature vector for each word of the second learning data generated in step S116. Learn parameters for each of domain features, other domain features, and common domain features.

  In step S120, a parameter for the common domain feature is extracted from the parameters learned in step S118, and the character string to be extracted is extracted and stored in the model storage unit 42 as a model parameter for assigning a label. , Output to the output unit 50, and the process ends.

  When extracting the character string to be extracted and assigning a label, each feature is extracted from the analysis target data, and each extracted feature and the character string to be extracted are extracted and assigned a label. The character string to be extracted is extracted from the data to be analyzed based on the model parameters to be applied, and a label is assigned.

  As described above, according to the model learning device according to the embodiment of the present invention, using the dictionary, the pseudo-correct label is assigned to the character string of the second learning data to which no label is attached, A feature is extracted for each word of the first learning data and each word of the second learning data, and for each word of the first learning data and each word of the second learning data, the target domain feature and other domain feature are extracted. , And generating a feature vector composed of common domain features, learning parameters for each of target domain features, other domain features, and common domain features, and extracting parameters for common domain features to extract character strings to be extracted It is possible to learn a model for giving a label with high accuracy.

  In addition, learning data can be expanded in a pseudo manner by using a dictionary, and learning data can be augmented by using a transfer learning method, so that a highly accurate character string extraction model can be obtained.

  In addition, by applying the model learned in the present invention to the specific expression extraction apparatus, it is possible to extract the specific expression from the document and assign an appropriate label.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the above embodiment, the specific expression extraction has been described as an example. However, the present invention can also be applied to prescriptive extraction. Just use a dictionary. Of course, by using learning data and a dictionary as a target, it is possible to apply to other than specific expression extraction and prescriptive extraction.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 30 Teacher database 32 Document collection database 34 Dictionary database 36 Dictionary collation part 38 Transfer learning feature generation part 40 Model learning part 42 Model storage part 50 Output part 100 Model learning apparatus

Claims (4)

An input unit that receives input of first learning data that is a document in which a label is previously assigned to a character string to be extracted; and second learning data that is a document to which the label is not attached;
A character string included in the document of the second learning data is collated with a dictionary in which the character string and the label are associated and stored, and the label is stored in the dictionary in the document of the second learning data. A dictionary collation unit that gives the label to a character string that matches the character string stored in association with
The feature is extracted from the first learning data to be a target domain feature, the target domain feature is duplicated to be a common domain feature, and a feature vector including the target domain feature, another domain feature, and the common domain feature is generated. The feature is extracted from the second learning data in which a label is assigned to the character string to be the other domain feature, the other domain feature is duplicated to be the common domain feature, and the feature vector is generated A generator,
Based on the feature vector of the first learning data and the feature vector of the second learning data, learn parameters for each of the target domain feature, the other domain feature, and the common domain feature, and the learning A learning unit that extracts a parameter for the common domain feature from the extracted parameters, extracts the extraction target character string, and outputs the extracted character string as a model parameter;
A model learning apparatus comprising: The character string to be extracted is a specific expression,
The label is a proper expression classification,
The model learning apparatus according to claim 1, wherein the model is a model for extracting the specific expression and assigning the specific expression classification to the specific expression. A step in which the input unit receives input of first learning data that is a document in which a label is pre-assigned to a character string to be extracted and second learning data that is a document to which the label is not attached; ,
A dictionary collation unit collates a character string included in the document of the second learning data with a dictionary in which the character string and the label are associated and stored, and in the document of the second learning data, Assigning the label to a character string that matches the character string stored in the dictionary in association with the label;
A feature generation unit extracts features from the first learning data to obtain target domain features, duplicates the target domain features into common domain features, and uses the target domain features, other domain features, and the common domain features. Generating a feature vector, extracting the feature from second learning data in which a label is assigned to the character string to obtain the other domain feature, duplicating the other domain feature to obtain the common domain feature, and the feature Generating a vector;
A learning unit learns parameters for each of the target domain feature, the other domain feature, and the common domain feature based on the feature vector of the first learning data and the feature vector of the second learning data. Extracting a parameter for the common domain feature from the learned parameters, extracting the character string to be extracted, and outputting the extracted parameter as a model parameter; and
Model learning method including   The program for functioning a computer as each part which comprises the model learning apparatus of Claim 1 or 2.