JP2009282686A - Apparatus and method for learning classification model - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To construct a highly precise classification model even when teacher data whose quality is bad are included. <P>SOLUTION: Coordinates corresponding to expert data in which the reliability of labeling satisfies a prescribed reference and non-expert data in which the reliability of labeling is not clear are acquired, and a distance between the non-expert data and the expert data is calculated, and a neighboring distance is defined by applying it to a prescribed rule. Then, the expert data within the range of the neighboring distance are retrieved from the selected non-expert data, and the same label probability is calculated, and applied to reliability function based on such probability that the applied label is matched with the label of the expert data within the range of the neighboring distance, and the reliability of the non-expert data is determined and added. Then, a classification model for labeling desired data is learnt based on the expert data and the non-expert data to which the reliability has been added. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a classification model learning apparatus and a classification model learning method for labeling classification target data in machine learning.

  One of the important fields in data mining is machine learning. Machine learning is often used for classification problems. In a classification problem, a computer builds a classification model by learning human evaluations. This application of machine learning has achieved many results in a wide range of fields such as image recognition, character recognition, and text classification. Such learning is generally called supervised learning.

  Supervised learning requires “teacher data” that teaches the computer the correct judgment, that is, data that has been “labeled” by human hands. The computer learns what classification should be performed based on the teacher data, and can automatically make judgments on new data. With the development of the IT environment, a large amount of detailed data is now available for machine learning, and it is expected that the use of these as teacher data will lead to the construction of a more accurate classification model.

  However, there is a problem with labeling when obtaining a “large amount of teacher data” from a “large amount of data”. In other words, in order to use the obtained data as teacher data, it is necessary to assign a label determined by a human to the data, but for accurate labeling, the domain from which the data was taken is used. Accurate judgment based on all knowledge and experience is essential.

  Ideally, it is desirable that the experts in the subject field satisfying these conditions perform the labeling, but it is very expensive to ask the experts to label all the data. However, since the cost is practically limited, when a large amount of teacher data is required, the teacher data labeled by a non-expert is used. This is because high-cost teacher data by experts tends to be small, whereas low-cost teacher data by non-experts can be acquired in a relatively large amount. On the other hand, teacher data by non-experts may contain a relatively large number of erroneous labels due to misjudgment and inaccuracy of knowledge.

  Moreover, in general machine learning, information on acquisition of teacher data is not used, and “good teacher data” such as labeling by experts and “teaching data including noise” such as labeling by non-experts are used. Even in a mixed situation, all data is handled in the same row and used equally for learning.

  Therefore, when a small amount of teacher data by experts and a large amount of teacher data by non-experts are considered to be the same as before and used for learning, the noise contained in the non-expert data greatly affects learning, and a highly accurate classification model is built. There are cases where this is not possible.

On the other hand, when learning a classification model, it is generally performed to selectively use a part of the teacher data and to perform a learning with a weight on a part of the teacher data. Yes. AdaBoost, which is one of the typical techniques of ensemble learning, is one of them. AdaBoost generates a learning device by giving weights to the learning data, and repeatedly generates a learning device by increasing the weight for the incorrectly classified data at that time, thereby generating a plurality of weak learning devices. This is a method of performing classification by weighted voting of those weak learners (see, for example, Patent Document 1 and Non-Patent Document 1).
JP 2002-133389 A Y. Freund and RE Shapire, “Experiments with a new boosting algorithm”, Proc. Of the 13th. Int. Conf. On Machine Learning, 1996, 148-156

  However, the conventional technology only applies data weights to teacher data in a form according to a predetermined algorithm, and does not include knowledge / information before starting the learning process of difference in accuracy of teacher data. However, different types of teacher data such as a small amount of teacher data by experts (hereinafter referred to as “expert data”) and a large amount of teacher data by non-experts (hereinafter referred to as “non-expert data”) are used as before. When used in the same row for learning, there is a problem in that noise included in inferior teacher data greatly affects learning, so that a highly accurate classification model cannot be constructed.

  In response to such a problem, the present applicant has proposed a technique for learning a classification model with high accuracy by using a small amount of teacher data by an expert in Japanese Patent Application No. 2007-278893. In this method, a classification model is learned by adding reliability to the label of non-expert teacher data based on teacher data from an expert and reflecting the reliability in learning the classification model. This reliability is obtained according to a distance (for example, Euclidean distance or cosine distance) between coordinates in which each of expert data and non-expert data is associated based on a predetermined rule.

  N pieces of expert data close to the target non-expert data are searched. If the labels are the same, the reliability is obtained from the expert data. This reliability is given, for example, in inverse proportion to the distance. If expert data near non-expert data has the same label, the non-expert data can obtain high reliability. It can be said that this indicates an intuitive reliability rating that the closer the reliable data is, the higher the reliability is.

  However, the above-described reliability rating method implicitly assumes that expert data does not include erroneous labels. If the expert data are all given appropriate labels, it can be expected that the reliability of the non-expert data given by referring to them will also be an appropriate value. On the other hand, when the expert data includes an erroneous label, such addition of reliability is not necessarily appropriate. FIG. 10 and FIG. 11 are diagrams for explaining the relationship of reliability for labeling expert data and labeling non-expert data.

In FIG. 10, the expert data X ₁ exists very close to some non-expert data x ₁ . Since this X ₁ is very close, the reliability of x ₁ is high if the labels of X ₁ and x ₁ are the same, and low if they are different. Here, it is assumed that the label that should be originally given to X ₁ and x ₁ is L ₁ . Assuming that the accurate label L ₁ is assigned to the expert data X ₁ , the reliability is high when L ₁ is assigned to the non-expert data x ₁ , and a different label L ₂ is assigned. In some cases, the reliability is low. This can be said to be appropriate reliability.

In Figure 11, consider a case where the label L ₂ erroneous expert data X ₁ is assigned. At this time, the reliability is low when the label L ₁ that should originally be given to the non-expert data x ₁ is low, and the reliability is high when the label L ₂ that should not be given on the contrary is given. turn into. This is clearly the opposite of proper confidence.

  That is, when an erroneous label is included in the expert data, an inappropriate reliability is added to the non-expert data, and the performance of the classification model generated reflecting the reliability is deteriorated. In reality, it is considered that a small amount of erroneous labels are also included in the expert data. Therefore, it is necessary to add robust reliability to the erroneous labels in the expert data.

  Therefore, in view of the problems of the prior art, the present invention provides a classification model learning device and a classification model learning method capable of constructing a high-precision classification model even in a situation where poor quality teacher data is included. For the purpose.

  The classification model learning device according to the present invention includes an expert data storage unit that stores, as expert data, teacher data whose labeling reliability in machine learning satisfies a predetermined criterion, and teacher data whose labeling reliability is unknown. A non-expert data storage unit for storing as non-expert data; and connected to the expert data storage unit and the non-expert data storage unit to obtain coordinates corresponding to the expert data and the non-expert data, respectively Each distance from the data to the expert data is calculated, and a neighborhood distance definition unit that defines the neighborhood distance by applying the calculated distance to a predetermined rule, and the label attached to the non-expert data is the neighborhood The extract is within distance A reliability function storage unit that stores a reliability function based on the probability of matching the label attached to the data, the neighborhood distance definition unit, the reliability function storage unit, the expert data storage unit, and the non-expert data The probability is calculated by searching the expert data within the range of the neighborhood distance from the selected non-expert data connected to a storage unit, and applying the calculated probability to the reliability function to determine the non-expert data. A reliability determination unit that determines the reliability of the labeling in the expert data; and the expert data storage unit and the reliability determination unit connected to the expert data and the non-expert data to which the reliability is added Learning the classification model for labeling the desired classification target data And having model learning unit and, a.

  The classification model learning method according to the present invention stores teacher data whose labeling reliability in machine learning satisfies a predetermined standard as expert data, and teacher data whose labeling reliability is unknown as non-expert data. A classification model learning method performed by a computer, wherein each of the expert data and the non-expert data is obtained with corresponding coordinates to calculate a distance from the non-expert data to the expert data. A neighborhood distance defining step of defining a neighborhood distance by applying a distance to a predetermined rule, a selection step of selecting non-expert data to which the reliability is to be added from the stored non-expert data, and the selected non-expert data Within the range of the neighborhood distance from expert data The expert data is searched to calculate a probability that the label attached to the non-expert data matches the label attached to the expert data, and the calculated probability is set to a predefined reliability function. A reliability determination step for determining the reliability of the labeling of the non-expert data by applying to the non-expert data to which the determined reliability is added and the labeling of desired data based on the expert data And a classification model learning step for learning a classification model for performing.

  ADVANTAGE OF THE INVENTION According to this invention, the classification model learning apparatus and classification model learning method which can construct | assemble an accurate classification model are provided even in the situation where poor quality teacher data are included.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the overall configuration of a classification model learning device 1 according to an embodiment of the present invention. As shown in the figure, the classification model learning device 1 according to the present embodiment includes an expert data storage unit 11, a non-expert data storage unit 12, a neighborhood distance definition unit 13, a reliability function storage unit 14, and a reliability determination unit. 15, a non-expert data storage unit 16 with reliability, a classification model learning unit 17, a classification target data storage unit 18, a prediction unit 19, and a result display unit 20.

  The expert data storage unit 11 is a storage device that stores expert data. "Expert data" refers to teacher data that has been labeled by a knowledgeable and experienced expert to classify data in machine learning and has high labeling accuracy (reliability). .

  The non-expert data storage unit 12 is a storage device that stores non-expert data. “Non-expert data” refers to teacher data that has been labeled but whose accuracy (reliability) is unclear.

  The neighborhood distance definition unit 13 is a program that calculates inter-coordinate distances from non-expert data to expert data, and defines a neighborhood distance that represents a range in which the similarity between data is equal to or greater than a reference value based on the inter-coordinate distance. is there. Here, a plurality of distances are selected from the calculated inter-coordinate distances based on a predetermined rule, and the neighborhood distance is calculated from the average value of these distances, but the calculation method is not limited to this.

  FIG. 2 is a diagram for specifically explaining the expert data and the non-expert data in two dimensions. In the figure, circles represent expert data, square marks represent non-expert data, and the color of each mark represents a label. These coordinates are obtained by converting each data based on a predetermined rule. For example, in the classification of e-mails, a feature word list is created in advance by analyzing a large number of junk mails, and the feature word list is compared with the words in the received mail body to perform coordinate processing. Specifically, the comparison result with the N words included in the feature word list is expressed as 1 when the received mail is included in the received mail, and as 0 when not included in the received mail, so that the received mail data is expressed in N-dimensional coordinates. (For example, (1, 0, 1,..., 1)). Here, for the sake of explanation, it is assumed that N-dimensional data obtained by converting mail data into coordinates is represented in a two-dimensional manner. That is, when the contents of the received mail body are close, the coordinates are close, so that it can be used for labeling whether or not the mail is spam.

Further, FIG. 2 shows that the neighborhood distance definition unit 13 selects non-expert data and sequentially obtains the distance from the selected non-expert data to each expert data. For example, in the case where a rule that sets the neighborhood distance as “the average value of the distance from the non-expert data to the fourth nearest expert data” is determined in advance, the distance r ₄ is obtained for each non-expert data, and the average value is obtained. Is calculated.

  The reliability function storage unit 14 is a storage device that stores a reliability function suitable for the classification problem. This reliability function is a function that defines the reliability based on the same label probability of expert data within the vicinity distance from non-expert data, and a plurality of such functions are prepared in advance corresponding to various classification problems. It is preferable. A specific definition method will be described later.

  The reliability determination unit 15 searches for expert data in the vicinity of the non-expert data based on the neighborhood distance defined by the neighborhood distance definition unit 13 and calculates the same label probability with the non-expert data. Is applied to the reliability function acquired from the reliability function storage unit 14 to determine the reliability of the non-expert data. In addition, as a method for selecting which function to use from among a plurality of reliability functions, a method for selecting based on information input from the input device (not shown) by the user at the time of model creation and a function to be used are set in advance. And how to keep it.

  The non-expert data storage unit 16 with reliability is a storage device that stores non-expert data to which reliability is given by the processing in the reliability determination unit 15 (hereinafter referred to as “non-expert data with reliability”).

  The classification model learning unit 17 is a program that learns a classification model using expert data and non-expert data with reliability.

  The classification target data storage unit 18 is a storage device that stores data to be newly classified, that is, data without a label (hereinafter referred to as “classification target data”).

  The prediction unit 19 is a program that labels the classification target data stored in the classification target data storage unit 18 using the classification model obtained by the classification model learning unit 17. Note that when AdaBoost is used, the classification method in the prediction unit 19 is the same as the general AdaBoost method, and thus the description thereof is omitted.

  The result display unit 20 is a display device such as a display that displays the prediction result in the prediction unit 19.

  Hereinafter, the operation of the classification model learning device 1 will be described with reference to the drawings. In the present embodiment, expert data and non-expert data will be specifically described as two-dimensional data. FIG. 3 is a flowchart showing a specific example of processing in the neighborhood distance defining unit 13.

  In S301, it is determined whether there is non-expert data that has not yet been selected. If all the non-expert data has been selected, the process proceeds to S305. On the other hand, if there is non-expert data that has not been selected, the process proceeds to S302.

In S302, one non-expert data that has not been selected from the non-expert data storage unit 12 is selected.
In S303, the distances from the selected non-expert data to all the expert data are calculated.

In S304, the distance from the selected non-expert data to the kth expert data is held in a buffer area (not shown). The optimum integer k varies depending on the problem, but here, the integer k is a value set in advance by the user. For example, the distance distribution calculated in S303 is analyzed, and the integer k can be set so that the distance from each non-expert data is within a predetermined range. Further, when it is desired to consider a plurality of neighborhood expert data when adding reliability, the integer k may be increased.
In S305, the average of all the distances held is taken, and the value is output to the reliability determination unit 15 as a neighborhood distance, and the process is terminated.

  With the above processing, the average distance to the kth expert data is obtained. If an integer k suitable for the problem is set, this distance can be considered as a typical value that defines the neighborhood.

  FIG. 4 is a flowchart illustrating a specific example of processing in the reliability determination unit 15. In S401, it is determined whether there is non-expert data to be selected. Here, the reliability is given to all the non-expert data, and if there is no non-expert data to be selected, the process ends. On the other hand, if there is non-expert data to which no reliability is assigned, the process proceeds to S402.

In S <b> 402, one non-expert data to which no reliability has been given yet is selected from the non-expert data storage unit 12. Here, it is assumed that the j-th non-expert data represented by the following formula (1) is selected. Note that x represents coordinates and y represents a label.

In S403, expert data included in the vicinity of the selected non-expert data is searched from the expert data storage unit 11 and held. In this example, the “neighbor” means a region in a circle with a radius r centered on the non-expert data represented by the above formula (1) using the neighborhood distance r defined in the neighborhood distance definition unit 13. Shall point to. Therefore, when the neighborhood distance r is 0.5, the expert data X _j1 of the following equation (2) is included in the neighborhood, but the expert data X _j2 of the equation (3) is not included in the neighborhood.

In S404, the label probability is calculated from the searched N expert data. In this example, the number of neighboring expert data having the same label as the target non-expert data x _j is K, and the same label probability P _j is defined by the following equation (4).
P _j = K / N (4)

  In step S405, the reliability of the label of the non-expert data is calculated using a reliability function that receives Expression (4). The reliability function can be in a form suitable for the classification problem. FIG. 5 to FIG. 7 are diagrams for explaining specific examples of the reliability function according to the evaluation criterion of the classification problem. In order to intuitively understand the nature of the reliability function, 10 examples of expert data are included in the vicinity of the non-expert data represented by the target equation (1), and 9 examples are inherent due to noise. Consider the situation where 8 cases have the same label.

Under this situation, for example, if the labeling of non-expert data matches 80% or more of the neighboring expert data, and if it is desired to give high reliability to the labeling, the following equation (5) It is preferable to use a reliability function. Here, a is a parameter that determines the shape of the function.

FIG. 5 is a diagram for explaining the reliability function of Expression (5). Here, the horizontal axis is the same label probability (P _j ), the vertical axis is the reliability (c _j ), and a curve obtained by connecting points obtained by Equation (5) when a = 2.0 is shown. . When the number of labels changes from 9 to 8 cases, the change in reliability c _j due to noise changes from C (9/10) ≈0.98 to C (8/10) ≈0.96, and the reliability c _{j increases} . The impact of is small. That is, even if the same label in 10 cases in the vicinity is in any of 9 cases and 8 cases, it is set to obtain a result that the reliability of the label of the non-expert data is maintained high, It can be said that it is an intuitively reasonable reliability function.

In addition, when it is desired to set a strict setting against mixing of erroneous labels, it is preferable to use a reliability function such as the following equation (6).

FIG. 6 is a diagram illustrating the reliability function of Expression (6). Here, the horizontal axis is the same label probability (P _j ), the vertical axis is the reliability (c _j ), and a curve obtained by connecting points obtained by Equation (6) when a = 5.0 is shown. . When using this function, the reliability is greatly reduced if there is even one incorrect label. For example, it is particularly useful in fields that require high reliability such as medical care.

Further, when it is desired to set a tolerance for mislabeling, it is preferable to use a reliability function such as the following equation (7).

FIG. 7 is a diagram illustrating the reliability function of Expression (7). Here, the horizontal axis is the same label probability (P _j ), the vertical axis is the reliability (c _j ), and a curve obtained by connecting points obtained by Equation (7) when a = 10.0 is shown. . When this function is used, the reliability does not drop significantly even if many erroneous labels are included, and the reliability gradually decreases as the number of erroneous labels increases.

In S406, the reliability c _j obtained in S405 is added to the target non-expert data and stored in the non-expert data storage unit 16 with reliability in the form of the following equation (8).

If it is an example of the above-described two-dimensional data (non-expert data of equation (1)), it is stored in the non-expert data storage unit 16 with reliability in the form of the following equation (9).

Since the reliability of the expert data is always 1, it can be considered that the expert data is stored in the expert data storage unit 11 in the form of the following equation (10) in a pseudo manner.

  In this way, by using a reliability function that considers the probability of the same label within the neighborhood distance, even if an erroneous label is given to the nearest expert data, if other labels are accurate, non-expert data Appropriate reliability can be added to. It can be said that such reliability is more robust against erroneous labeling of expert data than reliability based only on the distance between data.

  FIG. 8 is a flowchart showing a specific example of processing in the classification model learning unit 17. As long as the learning device reflects the reliability, any learning device can be considered to function. However, in consideration of the ease of incorporation of the reliability with respect to the data weight, the learning device is adapted to the AdaBoost method. It is assumed that processing is performed at Other methods such as bagging may be used.

In step S801, the read non-expert data with reliability and expert data are given equal data weights w _j according to the AdaBoost method. In the present invention, since the reliability c _j obtained by the reliability determination unit 15 is added to the teacher data in addition to the conventional data weight w _j in AdaBoost, the n non-expert data read here are The following equation (11) and N expert data are each processed in the form of the following equation (12).

In S802, the reliability c _j assigned to the non-expert data is reflected in the data weight. Here, the data weight w ′ _j reflecting the reliability c _j with respect to the data weight w _{j in} AdaBoost is set by the following equation (13).
w ′ _j = c _j w _j (13)

By setting in this way, even for non-expert data that is considered to have a large data weight w _j and a large influence on learning, if the reliability c _j of the non-expert data is low, the value of the data weight w ′ _j is small. Thus, the influence of the teacher data with low reliability c _j included in the non-expert data can be reduced in a natural manner.

In S803, it generates a weak learners using data weights w _'j obtained in S802. Various weak learners used in AdaBoost, such as a decision tree, can be considered.
In step S804, the cost function is updated according to the data weight and the performance of the weak learner in accordance with the AdaBoost algorithm.

In step S805, it is determined whether an end condition is satisfied. If it is determined that the end condition is satisfied, the process proceeds to S806. On the other hand, if it is determined that the termination condition is not satisfied, the process returns to S802. Note that an end condition in the general AdaBoost method is that the number of weak learners satisfies a predetermined number. For example, if the user sets to create 100 weak learners, the end condition is that S802 to S805 are repeated 100 times.
In S806, a strong learner, which is a highly accurate classification model, is generated by combining the generated weak learners, and the process ends.

  In this way, by using the knowledge before starting the learning process of the difference in the accuracy of the teacher data, giving reliability to the non-expert data and incorporating it into the learning of the classification model, there is little expert data. Can also obtain a highly accurate classification model.

  FIG. 9 is a flowchart illustrating a specific example of processing in the prediction unit 19. In S901, the presence / absence of classification target data in the classification target data storage unit 18 is determined. If it is determined that there is classification target data, the process proceeds to S902. On the other hand, if it is determined that there is no classification target data, the process is terminated.

In S902, one classification target data is selected from the classification target data storage unit 18.
In S903, the selected classification target data is applied to the classification model for labeling, and the process returns to S901. The processing from S901 to S903 is repeated until labeling is completed for all the classification target data.

  By configuring as described above, even when teacher data (expert data) regarded as highly reliable includes noise, the reliability function with the same label probability as input and the accuracy of each teacher data By using prior knowledge and adding reliability to non-expert data and incorporating it into classification model learning, a highly accurate classification model can be obtained.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined.

  In the above embodiment, text data such as mail data has been described as an example, but the type of target data is not limited to this. That is, it is possible to create a classification model by converting data such as image data and audio data into coordinates based on a predetermined rule. For example, when two-dimensional X-ray image data is divided into M rows and N columns and converted into data of 1 row and M × N columns, M × N-dimensional coordinates can be obtained. In this case, it is preferable that the color classification (for example, 16 levels of gray scale) in the image data is a matrix component. Then, X-ray image data determined (labeled) by an experienced doctor as expert data is used as expert data, and data determined by an inexperienced doctor as non-expert data. By doing so, a highly accurate classification model can be created.

The block diagram which shows the example of whole structure of the classification model learning apparatus 1 which concerns on one Embodiment of this invention. The figure which demonstrates expert data and non-expert data concretely in two dimensions. 10 is a flowchart showing a specific example of processing in the neighborhood distance defining unit 13. 10 is a flowchart showing a specific example of processing in the reliability determination unit 15. The figure explaining the specific example of the reliability function according to the evaluation criteria of a classification problem. The figure explaining the specific example of the reliability function according to the evaluation criteria of a classification problem. The figure explaining the specific example of the reliability function according to the evaluation criteria of a classification problem. 14 is a flowchart showing a specific example of processing in the classification model learning unit 17. 14 is a flowchart illustrating a specific example of processing in the prediction unit 19. The figure explaining the relationship of the reliability with respect to labeling of expert data and labeling of non-expert data. The figure explaining the relationship of the reliability with respect to labeling of expert data and labeling of non-expert data.

Explanation of symbols

1 ... Classification model learning device,
11 ... Expert data storage unit,
12 ... Non-expert data storage unit,
13 ... Neighborhood distance definition part,
14 ... reliability function storage,
15 ... reliability determination unit,
16 ... Non-expert data storage with reliability,
17 ... Classification model learning unit,
18 ... Classification target data storage unit,
19 ... prediction part,
20 ... Result display part.

Claims (10)

An expert data storage unit that stores, as expert data, teacher data in which the reliability of labeling in machine learning satisfies a predetermined criterion;
A non-expert data storage unit for storing teacher data with unknown reliability of labeling as non-expert data;
The expert data storage unit and the non-expert data storage unit are connected to obtain coordinates corresponding to the expert data and the non-expert data, respectively, and calculate distances from the non-expert data to the expert data, respectively. A neighborhood distance defining unit that defines the neighborhood distance by applying the calculated distance to a predetermined rule;
A reliability function storage unit that stores a reliability function based on a probability that the label attached to the non-expert data matches the label attached to the expert data within the range of the neighborhood distance;
Connected to the neighborhood distance definition unit, the reliability function storage unit, the expert data storage unit, and the non-expert data storage unit, and searches for the expert data within the neighborhood distance from the selected non-expert data And calculating the probability, and applying the calculated probability to the reliability function to determine the reliability of the labeling in the non-expert data;
A classification model that is connected to the expert data storage unit and the reliability determination unit and learns a classification model for labeling desired classification target data based on the expert data and non-expert data to which the reliability is added The learning department,
A classification model learning apparatus characterized by comprising:   The reliability function defines a relationship between a probability that the label attached to the non-expert data matches the label attached to the expert data within the vicinity distance and the reliability of the labeling. The classification model learning device according to claim 1, wherein:   The neighborhood distance defining unit calculates a distance between the coordinates from the non-expert data to the expert data, and ranks the non-expert data for each non-expert data. The classification model learning device according to claim 1 or 2, wherein the average value is calculated by summing up from each of the values, and the average value is defined as the neighborhood distance.   The classification model learning apparatus according to claim 1, wherein the reliability function is created in advance according to a desired evaluation criterion in a classification problem.   5. The classification model learning unit according to claim 1, wherein the classification model learning unit learns the classification model by reflecting the reliability added by the reliability determination unit with respect to data weight in ensemble learning. The classification model learning device according to claim 1. A classification model learning method performed by a computer that stores teacher data whose reliability of labeling in machine learning satisfies a predetermined standard as expert data and teacher data whose labeling reliability is unknown as non-expert data ,
Each of the expert data and the non-expert data obtains the corresponding coordinates, calculates the distance from the non-expert data to the expert data, and applies the calculated distance to a predetermined rule to determine the neighborhood distance. A neighborhood distance defining step to define;
A selection step of selecting non-expert data to be added with the reliability from the stored non-expert data;
The expert data within the range of the neighborhood distance is searched from the selected non-expert data, and the probability that the label attached to the non-expert data matches the label attached to the expert data is calculated. And a reliability determination step of applying the calculated probability to a predefined reliability function to determine the reliability of the labeling of the non-expert data;
A classification model learning step of learning a classification model for labeling desired data based on the non-expert data to which the determined reliability is added and the expert data;
A classification model learning method characterized by comprising:   The reliability function defines a relationship between a probability that the label attached to the non-expert data matches the label attached to the expert data within the vicinity distance and the reliability of the labeling. The classification model learning method according to claim 6, wherein:   In the neighborhood distance defining step, a distance between the coordinates from the non-expert data to the expert data is calculated and ranked for each non-expert data, and a distance for a desired rank is set as the distance of the non-expert data. 8. The classification model learning method according to claim 6 or 7, wherein the average value is calculated by summing up from each, and the average value is defined as the neighborhood distance.   The classification model learning method according to any one of claims 6 to 8, wherein the reliability function is created in advance according to a desired evaluation criterion in a classification problem.   10. The classification model learning step, wherein the classification model is learned by reflecting the reliability added in the reliability determination step with respect to the data weight in the ensemble learning. The classification model learning method as described in any one of Claims.

Images