JP2018041300A - Machine learning model generation device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform previous learning by automatically determining a class during the previous learning without requiring much time for searching for a condition of the previous learning when generating a model for additional learning.SOLUTION: An image reading device comprises: first similarity calculation means for calculating similarity between classes of which the relationship is predefined, based on the mutual relationship of the classes; second similarity calculation means for calculating similarity between the classes based on feature amounts of the classes; determination means for determining whether the classes are similar based on calculation results of the first calculation means and the second calculation means; class merging means for merging the classes that are determined similar by the determination means, into one class; and machine learning means for performing supervised machine learning processing.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a machine learning model generation device and a program.

  Specific expression extraction, which is also used as an elemental technology for applications using natural language processing, is a technique for extracting specific expressions such as proper nouns contained in text. Support Vector Machine (SVM) and Conditional Random Fields (CRFs) There is known a system using a machine learning technique that handles an identification problem such as a classification problem.

  Patent Document 1 below discloses a model parameter estimation method that can estimate a model parameter adapted to additional data while reducing the influence on parameters of an existing model.

  Non-Patent Document 1 below discloses Hidden-Unit CRF (HUCRF) which is a kind of derivation of CRFs in which a hidden unit layer is added to CRFs.

  In Non-Patent Document 2 below, HUCRF is made to learn temporary labeled data by clustering using canonical correlation analysis for an unlabeled data group, and the obtained parameters are actually labeled as initial values. A method of pre-learning for performing additional learning using data with a mark is disclosed.

Japanese Patent Laying-Open No. 2015-38709

L.maaten, M.welling, L.K.Saul "Hidden-Unit Conditional Random Fields" Int. Conf. On Artificial Intelligence & Statistics pp.479-488, 2011 Y.B.Kim, K.Stratos, R.Sarikaya, "Pre-training of Hidden-Unit CRFs" Association for Computational Linguistics, pp.192-198, 2015

  In the identification problem, generally, supervised learning is performed in which learning data (known data) to which a label (also referred to as a “tag” or the like) is already given is given and learning is performed, and the model parameters obtained as a result of learning are performed. Is used to identify unknown data (classification by assigning one of the labels). In order to achieve high accuracy using such a machine learning method, a large amount of learning data (corpus or dictionary) is generally required.

  For example, there are many documents in the company that contain highly confidential information, and there are many that cannot be understood unless the knowledge and rules unique to each company are assumed. For this purpose, a learning corpus needs to be prepared independently. Since it is extremely expensive to give a large amount of annotation (related annotation information) to such an in-company document, as a result, even if trying to extract a specific expression for an in-company document, high accuracy is achieved. It is not realistic to prepare as much learning data as can be achieved.

  Even in the case of tasks for which a large amount of learning data is not available, additional learning is performed with the target task using some model parameters (also referred to as “models”) that have been pre-trained in another task with a large amount of learning data as initial conditions This may improve the learning accuracy (if it is an identification problem, the accuracy of identifying unknown data using the obtained model), but appropriate pre-learning conditions that increase the accuracy Since it is unknown, it takes time to search for conditions.

  An object of the present invention is to provide a machine learning model generation apparatus and a program that can generate a model with higher learning accuracy compared to a case where class integration is not performed in advance learning.

[Model generator for machine learning]
The present invention according to claim 1 is a first similarity calculation means for calculating a similarity between the classes based on a relationship between classes in which a relationship is defined in advance.
Second similarity calculation means for calculating the similarity between the classes based on the feature amount of the class;
A determination unit that determines whether the classes are similar based on the calculation results of the first calculation unit and the second calculation unit;
Class integration means for integrating the classes determined to be similar by the determination means into one class;
Machine learning means for performing supervised machine learning processing;
Is a machine learning model generation device.

  The present invention according to claim 2 is the model generation apparatus for machine learning according to claim 1, wherein the relationship is a hierarchical structure or a graph structure.

  According to a third aspect of the present invention, the machine learning means comprises model parameter estimation means for estimating model parameters, and means for estimating posterior probabilities in each class of data from the learned model parameters. Item 3. The model generation device according to item 1 or 2.

[program]
The present invention according to claim 4 is a step of calculating a similarity between the classes as a first similarity based on a relationship between classes whose relationships are defined in advance.
Calculating a similarity between the classes as a second similarity based on the feature amount of the class;
Determining whether the classes are similar based on the first similarity and the second similarity;
Combining the classes determined to be similar into a single class;
Performing supervised machine learning processing;
Is a program for causing a computer to execute.

  According to the first aspect of the present invention, it is possible to generate a model with higher learning accuracy compared to a case where class integration is not performed in the prior learning.

  According to the second aspect of the present invention, in addition to the effect of the first aspect, the present invention can be applied to a case where the class relationship is a hierarchical structure or a graph structure.

  According to the present invention of claim 3, in addition to the effect of claim 1 or 2, it is determined whether or not the classes are similar based on the second similarity calculated based on the posterior probability. A machine learning model generation apparatus is obtained.

  According to the fourth aspect of the present invention, it is possible to generate a model with higher learning accuracy compared to a case where class integration is not performed during pre-learning.

It is a block diagram which shows the function structure of the model production | generation apparatus 10 which concerns on embodiment of this invention. It is a block diagram which shows the hardware constitutions of the model production | generation apparatus 10 which concerns on embodiment of this invention. It is a flowchart for demonstrating operation | movement of the model production | generation apparatus 10 which concerns on embodiment of this invention. The example of the hierarchical structure of the class which concerns on Example 1 is shown. It is an example of the estimated probability of each instance in the class according to the first embodiment. 3 is a matrix showing correlation coefficients between learning classes according to the first embodiment. The relationship between the similarity on the hierarchical structure between the learning classes based on Example 1, and the hierarchical structure of a class is shown. In Example 1, the hierarchical structure after class integration processing is shown. The example of the graph structure of the class which concerns on Example 2 is shown. It is an example of the estimated probability of each instance in the class according to the second embodiment. 10 is a matrix showing correlation coefficients between learning classes according to the second embodiment. The relationship between the similarity on the graph structure between the learning classes based on Example 2, and the graph structure of a class is shown. In Example 2, the graph structure after a class integration process is shown.

  FIG. 1 is a block diagram illustrating a functional configuration of a model generation device 10 according to an embodiment of the present invention. The model generation apparatus 10 according to the present embodiment includes a machine learning unit 11, a first similarity calculation unit 12, a second similarity calculation unit 13, a determination unit 14, a class integration unit 15, and an output unit 16. It is.

  In the machine learning unit 11, a machine learning process is performed from the learning data 17 in which labeled learning data is recorded, and a model parameter estimation and a class to which instance (learning data for each case) corresponds. Estimate posterior probabilities for classification.

  In the first similarity calculation unit 12, the vector amount generated based on the posterior probability of each instance estimated by the machine learning unit 11 (hereinafter also referred to as “estimated probability”) is used as the feature amount of each class. Based on the feature quantity, the similarity between the classes (“feature quantity similarity”) is calculated.

  The second similarity calculation unit 13 calculates the similarity (“relationship similarity”) between the classes based on the class relationship information 18 (for example, “class hierarchy” information). The initial value of the class relationship information is manually constructed based on the label of the learning data 16.

  The determination unit 14 determines that the class pair is “similar” when each of the feature amount similarity and the relationship similarity exceeds each predetermined threshold.

  The class integration unit 15 performs processing for integrating the class pairs determined as “similar” by the determination unit 14 into one class, and updates the information 18 on the relationship between the label of the learning data 17 and the class.

  The output unit 16 outputs the model parameter estimated by the machine learning unit 11 when there is no class pair determined as “similar” by the determination unit 14.

  FIG. 2 is a diagram illustrating a hardware configuration of the model generation device 10. The model generation device 10 includes a CPU 21, a memory 22, a storage device 23 such as a hard disk drive (HDD), and a display device 24, and these components are connected to each other via a control bus 25.

  The CPU 21 executes predetermined processing based on a control program stored in the memory 22 or the storage device 23 (or provided from a storage medium (not shown) such as a CD-ROM), and the model generating device 10 Control the behavior.

  In carrying out the present invention, the model generation device 10 may further include various input interface devices such as a keyboard and a touch panel.

  Next, the operation of the model generation apparatus according to the present embodiment will be described with reference to the flowchart of FIG. 3 taking as an example the case where the feature amount of each class is calculated based on the estimated probability of each instance. In the example shown below, when the relationship between the classes defined in advance has the hierarchical structure shown in FIG. 4 (Example 1), it has the graph structure shown in FIG. 9 (implementation). Example 2).

  In step S1, machine learning processing is executed in the lowest class (“corporate name”, “political organization name”, “prefectural name”, “city name”, “person name”) in the hierarchical structure, and then each instance An estimated probability is calculated for, and the process proceeds to step S2. Hereinafter, a case where the estimated probability of each instance becomes the value shown in FIG. 5 will be described as an example. (The estimated probability of the data of instance 1 is corporation name: 0.1, political organization name: 0.1, prefecture name: 0.4, municipality name: 0.3, person name: 0.1. )

  The feature quantity of each class can be calculated as, for example, a vector quantity having an estimated probability value of each instance as a value of each dimension.

  In step S2, after the feature amount of each learning class is generated from the estimated probability of each instance, a correlation coefficient between learning classes is calculated based on the generated feature amount of each learning class. In FIG. 6, it is expressed as a correlation matrix of feature amounts.

  In step S3, which can be performed in parallel with steps S1 and S2, the similarity in the hierarchical structure between each learning class is calculated based on the hierarchical structure of the class. For example, classes similar to each other in the same upper hierarchy (parent) (sibling classes) are set to a similarity of “1”, and other classes are set to a similarity of “0”. The degree can be calculated.

  Taking the hierarchical structure of FIG. 4 as an example, the combinations of “corporate name” and “political organization name” and “prefectural name” and “city name” are the same parent (“organization name”, “ Since the place name “) is the parent, the similarity in the hierarchical structure is“ 1 ”, and for combinations other than these, the similarity in the hierarchical structure is“ 0 ”. In FIG. 7, the similarity between each learning class is expressed as a matrix.

  After completion of steps S2 and S3, the process proceeds to step S4. In step S4, the correlation coefficient of the feature amount calculated in step S2 and the similarity in the hierarchical structure calculated in step S3 are each equal to or greater than a predetermined threshold. Learning class pairs are extracted as similar class pairs. For example, when the threshold value of the correlation coefficient of the feature amount is 0.5 and the threshold value of the similarity in the hierarchical structure is 0.5, the pair of “city name” and “prefecture name” is similar. It will be extracted as a class pair.

  In step S5 following step S4, it is determined whether or not there is a similar class pair. If there is a similar pair, the process proceeds to step S6, and there is no similar class pair. If yes, go to Step S7.

  In step S6, the similar class pairs extracted in step S4 are integrated into one class. Specifically, the information on the relationship between classes (hierarchical structure information) recorded as a hierarchical structure and the label of learning data are updated. If the pair of “city name” and “prefecture name” is similar, they are merged, and the merged class name (label) is, for example, an upper hierarchy name ( Example: “place name”) can be used (FIG. 8). After class integration processing, the process proceeds to steps S1 and S3.

  In steps S1 and S3 that have been advanced again, the machine learning process and the calculation of the similarity in the hierarchical structure between the learning classes are performed again based on the updated hierarchical structure information and learning data labels. As a result of the hierarchical structure update, “location name” was added and “city name” and “prefecture name” were deleted. As a result, the lowest class in the hierarchical structure is “corporate name”, “political organization name”, “ "Place name", "Person name".

  In this way, the loop is repeated until there are no similar pairs. Finally, in step S7, the model is output and the process ends.

  In the second embodiment, the task of estimating the author from the text of the novel is assumed, and the relationship between the authors defined from the teacher-friend / friend relationship has a graph structure as shown in FIG. This shows that there is a discipline or friendship between the classes (the name of the novelist) connected by a line).

  In step S1, after machine learning processing is executed in all classes constituting the graph structure, an estimated probability is calculated for each instance, and the process proceeds to step S2. Hereinafter, a case where the estimated probability of each instance becomes the value shown in FIG. 5 will be described as an example. (The estimated probability of instance 1 data is novelist A: 0.1, novelist B: 0.1, novelist C: 0.4, novelist D: 0.3, novelist E: 0.1. .)

  The feature quantity of each class can be calculated as, for example, a vector quantity having an estimated probability value of each instance as a value of each dimension.

  In step S2, after the feature amount of each learning class is generated from the estimated probability of each instance, a correlation coefficient between learning classes is calculated based on the generated feature amount of each learning class. In FIG. 11, it is expressed as a correlation matrix of feature amounts.

  In step S3, which can be performed in parallel with steps S1 and S2, the similarity on the graph structure between each learning class is calculated based on the class graph structure. For example, the similarity degree on the graph structure can be calculated by setting the similarity degree “1” between classes directly connected by a line and the similarity degree “0” between other classes.

  When the graph structure of FIG. 12 is taken as an example, for example, “Novelist A” has a similarity of “1” for the combination of “Novelist B”, “Novelist C” and “Novelist D”. And the degree of similarity with “Noveler E” is “0”. In FIG. 12, the similarity between each learning class is expressed as a matrix.

  After completion of steps S2 and S3, the process proceeds to step S4. In step S4, the correlation coefficient of the feature amount calculated in step S2 and the similarity on the graph structure calculated in step S3 are each greater than or equal to a predetermined threshold value. Learning class pairs are extracted as similar class pairs. As an example, when the threshold value of the correlation coefficient of the feature amount is 0.5 and the threshold value of the similarity on the graph structure is 0.5, the pair of “novelist C” and “novelist D” is similar. Will be extracted as a class pair.

  In step S5 following step S4, it is determined whether or not there is a similar class pair. If there is a similar pair, the process proceeds to step S6, and there is no similar class pair. If yes, go to Step S7.

  In step S6, the similar class pairs extracted in step S4 are integrated into one class. Specifically, the information on the relationship of the classes recorded as the graph structure (graph structure information) and the label of the learning data are updated. As an example, FIG. 13 shows a graph structure after the integration process when the pair of “novelist C” and “novelist D” is a similar class pair. As a class name (label) after integration, for example, a name obtained by combining both names before integration can be used. After class integration processing, the process proceeds to steps S1 and S3.

  In Steps S1 and S3 that have been advanced again, based on the updated graph structure information and the label of the learning data, the machine learning process and the calculation of the similarity on the graph structure between the learning classes are performed again.

  In this way, the loop is repeated until there are no similar pairs. Finally, in step S7, the model is output and the process ends.

  As described above, the present invention can be applied to a machine learning model generation device and a program.

DESCRIPTION OF SYMBOLS 10 Model production | generation apparatus 11 Machine learning part 12 1st similarity calculation part 13 2nd similarity calculation part 14 Judgment part 15 Class integration part 16 Output part 17 Learning data 18 Class relationship information 21 CPU
22 memory 23 storage device 24 display device 25 control bus

Claims (4)

First similarity calculation means for calculating the similarity between the classes based on the relationship between the classes in which the relationship is defined in advance;
Second similarity calculation means for calculating the similarity between the classes based on the feature amount of the class;
A determination unit that determines whether the classes are similar based on the calculation results of the first calculation unit and the second calculation unit;
Class integration means for integrating the classes determined to be similar by the determination means into one class;
Machine learning means for performing supervised machine learning processing;
A machine learning model generation device comprising:   The machine learning model generation device according to claim 1, wherein the relationship is a hierarchical structure or a graph structure.   The model generation apparatus according to claim 1 or 2, wherein the machine learning means comprises model parameter estimation means for estimating model parameters, and means for estimating posterior probabilities in each class of data from the learned model parameters. . Calculating a similarity between the classes as a first similarity based on a relationship between classes in which a relationship is defined in advance;
Calculating a similarity between the classes as a second similarity based on the feature amount of the class;
Determining whether the classes are similar based on the first similarity and the second similarity;
Combining the classes determined to be similar into a single class;
Performing supervised machine learning processing;
A program that causes a computer to execute.

Images