WO2022249392A1 - 情報処理装置、情報処理方法、及びプログラム - Google Patents

情報処理装置、情報処理方法、及びプログラム Download PDF

Info

Publication number
WO2022249392A1
WO2022249392A1 PCT/JP2021/020174 JP2021020174W WO2022249392A1 WO 2022249392 A1 WO2022249392 A1 WO 2022249392A1 JP 2021020174 W JP2021020174 W JP 2021020174W WO 2022249392 A1 WO2022249392 A1 WO 2022249392A1
Authority
WO
WIPO (PCT)
Prior art keywords
training
examples
machine learning
artificial
information processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/020174
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
優太 畠山
穣 岡嶋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to PCT/JP2021/020174 priority Critical patent/WO2022249392A1/ja
Priority to US18/561,354 priority patent/US20240242123A1/en
Priority to JP2023523860A priority patent/JP7571879B2/ja
Publication of WO2022249392A1 publication Critical patent/WO2022249392A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Definitions

  • the present invention relates to technology for generating examples used in machine learning.
  • Non-Patent Document 1 the minority class instance (training example) closest to the decision boundary of the Support Vector Machine and the neighboring minority class instance are synthesized to create a minority class virtual instance. It is described to generate
  • Non-Patent Document 1 has the problem that virtual instances (artificial examples) are generated near the decision boundary, and artificial examples are not generated in regions other than near the decision boundary where training examples are insufficient. rice field.
  • One aspect of the present invention has been made in view of the above problems, and an example of its purpose is to provide a technique capable of generating artificial examples in areas where training examples used for machine learning training are lacking. That is.
  • An information processing apparatus provides a machine learning model group including a plurality of acquisition means for acquiring a plurality of training examples, and a plurality of machine learning models for outputting a prediction result from input of the training examples.
  • a selection means for selecting, from among the plurality of training examples, a training example having variations in a plurality of prediction results obtained using the machine learning model group after training; and the plurality of training generating means for generating artificial examples by synthesizing two or more of the training examples including the selected training example.
  • An information processing method includes obtaining a plurality of training examples, and generating a machine learning model group including a plurality of machine learning models that take the training examples as input and output a prediction result using the plurality of training examples. training, selecting, from among the plurality of training examples, a training example in which a plurality of prediction results obtained using the group of machine learning models after training vary, and among the plurality of training examples, Combining two or more training examples containing the selected training examples to generate an artificial example.
  • a program according to one aspect of the present invention is a program for causing a computer to function as an information processing apparatus, the computer comprising an acquisition unit that acquires a plurality of training examples, and a machine that receives the examples as input and outputs a prediction result.
  • training means for training a machine learning model group including a plurality of learning models using the plurality of training examples; and a plurality of prediction results obtained by using the machine learning model group after training among the plurality of training examples.
  • selection means for selecting a training example with variation in the training examples; and generation means for generating an artificial example by synthesizing two or more training examples including the selected training example from among the plurality of training examples. .
  • artificial examples can be generated in areas lacking training examples used for machine learning training.
  • FIG. 1 is a block diagram showing the configuration of an information processing device according to exemplary Embodiment 1 of the present invention
  • FIG. FIG. 3 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 1 of the present invention
  • FIG. 4 is a diagram schematically showing a specific example of an information processing method according to exemplary embodiment 1 of the present invention
  • FIG. 2 schematically illustrates an artificial example generated by exemplary embodiment 1 of the present invention
  • FIG. 7 is a block diagram showing the configuration of an information processing apparatus according to exemplary Embodiment 2 of the present invention
  • FIG. 7 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 2 of the present invention
  • FIG. 7 is a diagram schematically showing a specific example of an information processing method according to exemplary embodiment 2 of the present invention
  • FIG. 10 is a flow diagram showing the flow of a first generation process according to exemplary embodiment 2 of the present invention
  • FIG. 11 is a flow diagram showing the flow of a second generation process according to exemplary embodiment 2 of the present invention
  • FIG. 11 is a flow diagram showing the flow of a third generation process according to exemplary embodiment 2 of the present invention
  • FIG. 11 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 3 of the present invention
  • FIG. 11 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 4 of the present invention
  • FIG. 11 is a diagram schematically illustrating an information processing method according to exemplary embodiment 5 of the present invention
  • FIG. 4 is a diagram schematically explaining an artificial example generated by the technique described in Non-Patent Document 1
  • 1 is a block diagram showing the configuration of a computer functioning as an information processing device according to exemplary embodiments 1 to 5 of the present invention
  • FIG. 4 is a diagram schematically explaining an artificial example generated by the technique described in Non-Patent Document 1
  • 1 is a block diagram showing the configuration of a computer functioning as an information processing device according to exemplary embodiments 1 to 5 of the present invention
  • FIG. 1 is a block diagram showing the configuration of an information processing device 10. As shown in FIG.
  • the information processing device 10 is a device that generates artificial examples from a plurality of training examples using a machine learning model group.
  • the information processing device 10 includes an acquisition unit 11, a training unit 12, a selection unit 13, and a generation unit 14, as shown in FIG.
  • the acquisition unit 11 is an example of a configuration that implements the acquisition means described in the claims.
  • the training unit 12 is an example of a configuration that implements training means described in the claims.
  • the selection unit 13 is an example of a configuration that implements the selection means described in the claims.
  • the generation unit 14 is an example of a configuration that implements the generation means described in the claims.
  • the acquisition unit 11 acquires a plurality of training examples.
  • the training unit 12 trains a machine learning model group including a plurality of machine learning models that output prediction results using examples as input, using a plurality of training examples.
  • the selection unit 13 selects, from among the plurality of training examples, training examples in which there are variations in the plurality of prediction results obtained using the machine learning model group after training.
  • the generating unit 14 generates an artificial example by synthesizing two or more training examples including the selected training example from among the plurality of training examples.
  • a machine learning model group includes a plurality of machine learning models. Each machine learning model takes an example as an input and outputs a prediction result.
  • a prediction result may include, for example, a predicted probability that each of a plurality of labels is predicted. In this case, the label with the highest prediction probability may be referred to as the prediction result.
  • a machine learning model is, by way of example, a model generated using a machine learning algorithm such as a decision tree, neural network, random forest, or support vector machine. However, the machine learning algorithms used to generate each machine learning model are not limited to these.
  • the multiple machine learning models may all be models generated using the same machine learning algorithm. At least two of the plurality of machine learning models may be models generated using different machine learning algorithms.
  • the machine learning model group may be stored, for example, in the memory of the information processing device 10 or may be stored in another device communicably connected to the information processing device 10 .
  • the machine learning model group does not necessarily have to be "training target machine learning models to be trained using generated artificial examples”.
  • the group of machine learning models may include some or all of the machine learning models to be trained.
  • the machine learning model group does not have to include the machine learning model to be trained.
  • the number of machine learning models to be trained may be plural or singular.
  • An example is information that is input to each machine learning model and includes features. In other words, examples exist in feature space.
  • a training example is an example used to train a set of machine learning models. The training examples may be examples obtained by observation or may be artificial examples that are artificially generated.
  • a training example in which a plurality of prediction results vary is a training example in which the evaluation result of variation indicates that the variation is large.
  • evaluation of variability means evaluating whether or not the variability of a plurality of prediction results is large.
  • the variability evaluation may be an evaluation based on the entropy of voting results. The entropy of voting results will be described in detail in exemplary embodiment 2 below.
  • the evaluation of variation may be an evaluation based on the ratio of prediction results showing the same label among a plurality of prediction results.
  • the evaluation of variation is not limited to the one described above.
  • a training example evaluated as having a large variation in a plurality of prediction results is also referred to as a “training example with a variation in prediction results”.
  • a training example evaluated as not having a large variation in a plurality of prediction results is also described as a “training example having a small variation in prediction results”.
  • FIG. 2 is a flow diagram showing the flow of the information processing method S10. As shown in FIG. 2, the information processing method S10 includes steps S101 to S104.
  • step S101 acquisition process
  • the acquisition unit 11 acquires a plurality of training examples.
  • the acquisition unit 11 may acquire a plurality of training examples by reading them from memory.
  • the acquisition unit 11 may acquire a plurality of training examples from an input device or from a device connected via a network.
  • the plurality of training examples obtained in this step includes one or both of observed examples and artificial examples.
  • step S102 training process
  • the training unit 12 trains a group of machine learning models using a plurality of training examples acquired in step S101.
  • the training examples used for training each machine learning model group may be a part or all of the plurality of training examples acquired in step S101.
  • step S103 In step S ⁇ b>103 (selection processing), the selection unit 13 selects training examples having variations in prediction results from among the plurality of training examples.
  • the selection unit 13 may select one such training example, or may select a plurality of such training examples.
  • the selection unit 13 inputs a training example to be evaluated among a plurality of training examples to each machine learning model after training, and acquires a prediction result output from each machine learning model. Thereby, the selection unit 13 obtains a plurality of prediction results for the training example to be evaluated. In addition, the selection unit 13 evaluates variations in the obtained prediction results. If the selection unit 13 evaluates that a plurality of prediction results have large variations, the selection unit 13 selects the training example as "a training example having variations in prediction results."
  • the selection unit 13 makes some or all of the plurality of training examples subject to variation evaluation. For example, when each machine learning model group is trained using a part of a plurality of training examples, the selection unit 13 selects the other part of the plurality of training examples (i.e., the machine learning model group was not used for training training examples) may be evaluated.
  • step S104 generation processing
  • the generation unit 14 generates an artificial example by synthesizing two or more training examples including the training example selected in step S103 among the plurality of training examples. For example, the generation unit 14 may synthesize the selected training example with other training examples that exist nearby in the feature space. Further, for example, when a plurality of training examples are selected in step S103, the generation unit 14 may synthesize the selected training examples.
  • the generating unit 14 may generate one artificial example by synthesizing two training examples, or may generate one artificial example by synthesizing three or more training examples. In this step, the generation unit 14 may generate one artificial example, or may generate a plurality of artificial examples.
  • the synthesizing process performed by the generating unit 14 is represented by the following equation (1) as an example.
  • Equation (1) ⁇ xv represents an artificial example, and x i represents a training example selected by the selection unit 13 .
  • x j may be another training example selected by the selection unit 13, or may be another training example that is not selected. If it is another training example that has not been selected, then x j is a training example that is in the neighborhood of x i .
  • ⁇ is a weighting coefficient that satisfies 0 ⁇ 1.
  • the generation unit 14 determines the value of the coefficient ⁇ using random numbers generated by a random function. Note that the synthesizing process performed by the generating unit 14 is not limited to the method described above, and the generating unit 14 may synthesize a plurality of training examples by other methods.
  • FIG. 3 is a diagram schematically showing a specific example of the information processing method S10.
  • the plurality of training examples T acquired by the acquisition unit 11 in step S101 include training examples t1, t2, t3, .
  • the machine learning model group trained by the training unit 12 in step S102 includes machine learning models m1, m2, m3, . Each of the machine learning models m1, m2, m3, . Each of the machine learning models m1, m2, m3, .
  • the selection unit 13 evaluates variations in a plurality of prediction results for training examples t1 to t10 to be evaluated.
  • solid-line circles indicate training examples with variability in prediction results
  • dashed-line circles indicate training examples with small variability in prediction results.
  • a plurality of prediction results obtained from machine learning models m1, m2, m3, . is evaluated.
  • training examples t6, t9, and t10 a plurality of prediction results obtained from machine learning models m1, m2, m3, .
  • training examples t3 and t4 two of a plurality of prediction results obtained from machine learning models m1, m2, m3, . . . are "A" and one is "B.” is evaluated as large.
  • training examples t7 and t8 two of a plurality of prediction results obtained from machine learning models m1, m2, m3, . . . are "B" and one is "A.” is evaluated as large.
  • the selection unit 13 selects training examples t3, t4, t7, and t8 with variations in prediction results.
  • the generation unit 14 generates an artificial example t51 by synthesizing the training example t3 with variations in prediction results and the neighboring training example t5. Further, the generation unit 14 generates an artificial example t52 by synthesizing the training example t4 in which the prediction results vary and the neighboring training example t1. Further, the generation unit 14 generates an artificial example t53 by synthesizing a plurality of training examples t7 and t8 having different prediction results. In FIG. 3, the double-lined circle indicates an artificial example.
  • the exemplary embodiment obtains a plurality of training examples, trains a machine learning model family including a plurality of machine learning models that take the examples as input and output a prediction result using the plurality of training examples, and trains the plurality of training examples. From among the training examples, select training examples with variations in the prediction results obtained using the machine learning model group after training, and synthesize two or more training examples including the selected training examples from among the plurality of training examples. and to generate an artificial example.
  • training examples with multiple prediction results that vary are considered to be in areas where there are insufficient training examples in the feature space.
  • Artificial examples obtained by synthesizing multiple training examples containing such training examples are likely to be generated in areas lacking training examples.
  • the present exemplary embodiment can generate artificial examples in regions lacking training examples.
  • FIG. 4 is a diagram schematically illustrating an artificial example generated by this exemplary embodiment.
  • FIG. 14 is a diagram schematically explaining an artificial example generated by the technique described in Non-Patent Document 1.
  • FIG. 4 and 14 solid-line circles indicate training examples with small variations in prediction results, dashed-line circles indicate training examples with variations in prediction results, and double-line circles indicate artificial examples. show.
  • Regions R1, R2, and R3 indicate regions in the feature amount space. Regions R1, R2, and R3 contain artificial examples with varying prediction results, and are regions lacking artificial examples.
  • Non-Patent Document 1 As shown in FIG. 14, the technique described in Non-Patent Document 1 generates an artificial example in a region R1 near the decision boundary B by the support vector machine. However, with the technique described in Non-Patent Document 1, it is difficult to generate artificial examples in regions R2 and R3 other than the vicinity of the decision boundary B, where training examples are insufficient.
  • the exemplary embodiment generates artificial examples by synthesizing multiple training examples, including training examples with varying prediction results.
  • the present exemplary embodiment can generate artificial examples in regions R1, R2, R3 lacking training examples.
  • this exemplary embodiment can suppress the generation of artificial examples biased toward a partial region R1.
  • this exemplary embodiment employs a configuration in which a group of machine learning models are used to select training examples with varying prediction results.
  • this exemplary embodiment suppresses the generation of artificial examples in biased regions compared to the case of generating artificial examples in the vicinity of the decision boundary like the technique described in Non-Patent Document 1. be able to.
  • This also allows the present exemplary embodiment to generate artificial examples in regions that are more deficient in training examples compared to using prediction probabilities to select training examples with varying prediction results. be able to. This is because, for example, when a decision tree is included in a group of machine learning models, the decision tree may make an erroneous prediction with a prediction probability of 1.
  • FIG. 5 is a block diagram showing the configuration of the information processing device 20.
  • the information processing device 20 is a device that generates an artificial example from a plurality of examples using the machine learning model group COM0.
  • the machine learning model group COM0 is configured almost similarly to the machine learning model group in the first exemplary embodiment. However, in this exemplary embodiment, at least two of the plurality of machine learning models included in the machine learning model group COM0 were generated using different machine learning algorithms.
  • the machine learning model group COM0 includes machine learning models that are training targets to be trained using artificial examples.
  • At least one machine learning model included in the machine learning model group COM0 is a decision tree.
  • the machine learning model being trained is a decision tree.
  • the information processing device 20 includes, as shown in FIG. .
  • the acquisition unit 21 is an example of a configuration that implements the acquisition means described in the claims.
  • the training unit 22 is an example of a configuration that implements training means described in the claims.
  • the selection unit 23 is an example of a configuration that implements the selection means described in the claims.
  • the generation unit 24 is an example of a configuration that implements the generation means described in the claims.
  • the labeling unit 25 is an example of a configuration that implements the labeling means described in the claims.
  • the output unit 26 is an example of a configuration that implements the output means described in the claims.
  • the acquisition unit 21 is configured in the same manner as the acquisition unit 11 in the first exemplary embodiment.
  • the training unit 22 is configured almost similarly to the training unit 12 in exemplary embodiment 1, but differs in that the machine learning model group COM0 is divided into a plurality of groups for training. Details of the training process by the training unit 22 will be described later.
  • the selection unit 23 is configured in substantially the same manner as the selection unit 23 in exemplary embodiment 1, but differs in the details of the training examples to be evaluated for variations in prediction results. The details of the training examples to be evaluated will be described later.
  • the generating unit 24 generates artificial examples by synthesizing the training examples selected by the selecting unit 23 and examples existing in the vicinity of the selected training examples in the feature space. As an example, the generator 24 generates an artificial example by synthesizing two training examples according to Equation (1) above.
  • the labeling unit 25 labels some or all of the training examples and artificial examples.
  • the label assigning unit 25 may assign a label based on information output from an input device that accepts user operations. Further, as an example, the labeling unit 25 may give labels obtained by inputting training examples and artificial examples to a machine learning model trained to output labels with examples as input.
  • the machine learning model that outputs the label is a machine learning model different from each machine learning model included in the machine learning model group COM0.
  • the machine learning model that outputs the label is preferably a model with higher prediction accuracy than at least one machine learning model included in the machine learning model group. For example, if the machine learning model to be trained included in the machine learning model group is a decision tree, the machine learning model that outputs the label may be a random forest.
  • the output unit 26 outputs the artificial example generated by the generation unit 24.
  • the output unit 26 may store the artificial example generated by the generation unit 24 in a recording medium such as an external storage device.
  • the output unit 26 may output the artificial example to an output device such as a display device, for example.
  • the control unit 27 controls each unit of the information processing device 20 .
  • the control unit 27 particularly adds the artificial examples generated by the generation unit 24 to a plurality of training examples, and causes the acquisition unit 21, the training unit 22, the selection unit 23, and the generation unit 24 to function again. .
  • FIG. 6 is a flow diagram showing the flow of the information processing method S20.
  • step S201 acquisition process
  • the acquisition unit 11 acquires a plurality of training examples.
  • the plurality of training examples to be acquired may include examples obtained by observation, or may include artificial examples.
  • Step S202 In step S ⁇ b>202 , the label assigning unit 25 assigns a label to each of the plurality of training examples acquired by the acquiring unit 21 .
  • step S203 training process
  • the training unit 22 trains each of the multiple machine learning model groups using some or all of the multiple training examples acquired by the acquisition unit 21 .
  • the details of the training process for training each machine learning model group will be described later.
  • step S204 In step S ⁇ b>204 (selection processing), the selection unit 23 selects one or more training examples with variations in prediction results from among the plurality of training examples acquired by the acquisition unit 21 .
  • the selection processing performed by the selection unit 23 will be described later.
  • the details of the process of selecting training examples with varying prediction results will be described later.
  • step S205 generation processing
  • the generation unit 24 identifies a plurality of training examples including the training example selected by the selection unit 23 as synthesis targets.
  • the generation unit 24 also generates an artificial example by synthesizing a plurality of training examples specified as synthesis targets. The details of the generation process performed by the generation unit 24 will be described later.
  • step S206 the label assigning unit 25 assigns a label to each of the artificial examples generated by the generating unit 24.
  • FIG. in step S207 the control unit 27 determines whether to end the training process. As an example, when the number of times the processes of steps S203 to S206 have been executed is equal to or greater than a predetermined threshold, the control unit 27 determines to end the training process. On the other hand, when the number of times the processes of steps S203 to S206 have been executed is less than the predetermined threshold, it is determined that the training process is not finished. If the training process is not to end (NO in step S207), the control unit 27 proceeds to the process of step S208. On the other hand, when ending the training process (YES in step S207), the control unit 27 proceeds to the process of step S209.
  • Step S208 the control unit 27 adds one or more artificial examples generated in step S206 executed so far to a plurality of training examples. After completing the process of step S208, the control unit 27 returns to the process of step S203. In other words, the control unit 27 adds the artificial example to the plurality of training examples and causes the acquisition unit 21, the training unit 22, the selection unit 23, and the generation unit 24 to function again.
  • step S209 the output unit 26 outputs one or more artificial examples generated in step S206 executed so far.
  • One or more artificial examples thus generated using information processing method S20 are used to train a machine learning model to be trained.
  • the training unit 22 may perform the process of training the machine learning model to be trained.
  • FIG. 7 is a diagram schematically showing a specific example of the information processing method S20.
  • each divided group will be referred to as a machine learning model group COMi.
  • a plurality of machine learning models included in the machine learning model group COMi are hereinafter referred to as machine learning models mi-j.
  • the plurality of machine learning models mi-j included in the machine learning model group COMi may all be models generated by the same machine learning algorithm, or at least two of them may be models generated by mutually different machine learning algorithms. It can be a model.
  • the training unit 22 extracts the training example group Di from the training example group T acquired by the acquisition unit 21 in step S201.
  • Training example group Di is part of training example group T.
  • the training unit 22 may extract the training example group Di by random sampling.
  • the training example groups Di1 and Di2 may contain all the same training examples, or some or all of them may be different.
  • the selection unit 23 evaluates the variability of each training example to be evaluated, using an index of vote entropy in the QBC (query by committee) method.
  • the following equation (2) is an equation that indicates the training example ⁇ x for which the entropy of the voting result is the maximum.
  • C represents the total number of machine learning models mi-j in the machine learning model group COMi.
  • V(y) indicates the number of machine learning models mi-j that have predicted the label y in the machine learning model group COMi.
  • the selection unit 23 may select the training example ⁇ x indicated by the equation (2) as a training example having variations in prediction results. In this case, the number of training examples with variations in prediction results selected by the selection unit 23 for each machine learning model group COMi is one.
  • the selection unit 23 uses the M machine learning model group COMi to select M training examples with varying prediction results. For each machine learning model group COMi, the selection unit 23 may select a predetermined number of training examples in descending order of entropy of the voting result, or may select training examples whose entropy of the voting result is equal to or greater than a threshold. good. In this case, the number of training examples with variations in prediction results selected by the selection unit 23 for each machine learning model group COMi can be plural. In other words, in this case, the selection unit 23 selects M or more training examples with variations in prediction results using the M machine learning model group COMi. Further, the selection unit 23 may randomly select one or a predetermined number from among the M or more training examples with variations in the prediction results selected in this way, or the entropy of the voting results may be One or a predetermined number may be selected in descending order.
  • step S205 the generation unit 24 performs any one of the first generation processing S30, the second generation processing S40, and the third generation processing S50 using training examples having variations in the prediction results, thereby generating artificial training examples.
  • the first generation process S30 is a process of generating an artificial example by synthesizing training examples having variations in prediction results and neighboring training examples.
  • the second generation process is a process of generating an artificial example by synthesizing two or more training examples with varying prediction results.
  • the third generation process is a process of selectively executing either the first generation process or the second generation process.
  • the first generation processing S30 will be described with reference to FIG.
  • FIG. 8 is a flowchart showing the flow of the first generation processing S30.
  • the first generation process S30 includes steps S301 and S302.
  • step S204 previously performed, one or a plurality of training examples with varying prediction results are selected.
  • the generation unit 24 executes the following steps S301 and S302 for each of one or more training examples (hereinafter referred to as the training examples) having variations in prediction results.
  • Step S301 the generating unit 24 selects training examples near the training example.
  • Neighboring training examples may be training examples with varying prediction results or training examples with less variation in prediction results.
  • the neighboring training example may be a training example in the training example group T that is closest to the training example in the feature space.
  • the neighboring training example may be a training example whose distance in the feature space from the training example in the training example group T is equal to or less than a threshold.
  • Step S302 the generating unit 24 generates an artificial example by synthesizing the training example and the neighboring training example selected in step S301.
  • an artificial example tv1-1 is generated by synthesizing a training example t1 with variations in the prediction results and neighboring training examples.
  • an artificial example tv1-2 is generated by synthesizing the training example t2 with variations in prediction results and its neighboring training examples.
  • the generation unit 24 may use the above formula (1) as an example of synthesis processing.
  • the generating unit 24 may use known techniques such as MUNGE (see reference 1) and SMOTE (see reference 2) as other examples of combining processing.
  • FIG. 9 is a flowchart for explaining the flow of the second generation processing S40.
  • the second generation process S40 includes steps S401 and S402.
  • the second generation process can be executed when a plurality of training examples with different prediction results are selected in step S204.
  • the generation unit 24 executes the following steps S401 and S402 for each of a plurality of training examples (hereinafter referred to as training examples) having variations in prediction results.
  • Step S401 the selection unit 23 selects another training example different from the training example from among the plurality of training examples having variations in prediction results.
  • the selection unit 23 selects two or more training examples having a plurality of variations in prediction results from among the plurality of training examples.
  • the selection unit 23 may randomly select such other training examples from a plurality of training examples with varying prediction results.
  • the selection unit 23 selects, as such another training example, one having the smallest distance in the feature amount space from the training example among a plurality of training examples having variations in prediction results, or a distance equal to or less than a threshold. can be selected. Note that if the training example has already been used for synthesis, the processing of steps S401 and S402 relating to the training example need not be executed.
  • Step S402 the generation unit 24 generates an artificial example by synthesizing the training example and the other training example selected in step S401.
  • the artificial example tv2-1 is generated by synthesizing the training examples t11 and t12 with different prediction results.
  • the two or more training examples synthesized by the generation unit 24 may be selected using the same machine learning model group COMi as in this example, or at least one of them may be different from the others. It may be selected using the machine learning model group COMi.
  • the generator 24 selects two or more training examples from training examples t1, t2, . . . , t11, t12, .
  • step S402 It may be synthesized to produce artificial examples tv1-1, tv1-2, tv2-1, or tv2-2. Note that the technique used in the combining process in step S402 is the same as described in step S302, so detailed description will not be repeated.
  • FIG. 10 is a flowchart for explaining the flow of the third generation process S50.
  • the third generation process S50 includes steps S501 to S503.
  • step S204 one or more training examples with varying prediction results are selected.
  • the generation unit 24 executes the following steps S501 to S503 for each of one or more training examples (hereinafter referred to as the training examples) having variations in prediction results.
  • Step S501 the generator 24 selects either the first generation process or the second generation process.
  • the generator 24 may select the first generation process using a probability p determined by a random function, and select the second generation process if the first generation process is not selected.
  • the method of selecting either the first generation process or the second generation process is not limited to the method using the probability p, and may be another method.
  • Step S502 the generation unit 24 determines which one is selected. When the first generation process is selected, the generation unit 24 proceeds to the process of step S503 and executes the first generation process. On the other hand, when the second generation process is selected, the generation unit 24 proceeds to the process of step S504 and executes the second generation process. The details of the first generation process and the second generation process are as described above.
  • This exemplary embodiment has a configuration of executing a first generation process for generating an artificial example by synthesizing a training example with a variation in prediction results and instruction examples in the vicinity thereof.
  • the artificial examples generated by the first generation process are generated in the vicinity of the training examples in which the prediction results vary.
  • Training examples with varying prediction results are considered to be regions in the feature space that are deficient in training examples.
  • such artificial examples are generated in regions lacking training examples.
  • this exemplary embodiment has a configuration of executing a second generation process of synthesizing two or more training examples having variations in prediction results to generate an artificial example.
  • the artificial examples generated by the second generation process are obtained by synthesizing training examples in a region lacking training examples. Therefore, areas where such artificial examples exist are also likely to be deficient in training examples.
  • this exemplary embodiment has a configuration in which either the first generation process or the second generation process is selected and executed to execute the third generation process for generating an artificial example.
  • the artificial example generated by the third generation process is generated by the first generation process or the second generation process.
  • the region in which the artificial example is generated by the first generation process and the region in which the artificial example is generated by the second generation process can be different. Therefore, when a plurality of artificial examples are generated by the third generation process, there is a high possibility that these artificial examples will be generated distributed over more diverse regions lacking training examples.
  • the exemplary embodiment generates artificial examples that are biased toward regions where training examples are sufficient by performing any of the first generation process, the second generation process, and the third generation process. Instead, artificial examples can be generated in areas where training examples are more scarce.
  • the group of machine learning models includes a machine learning model to be trained. This allows the exemplary embodiments to generate artificial examples that are more effective at improving the accuracy of the machine learning model under training.
  • this exemplary embodiment employs a configuration in which at least two of the machine learning model groups are models generated by mutually different machine learning algorithms.
  • the machine learning model to be trained is a decision tree and not a support vector machine.
  • such training target machine learning models are included in the machine learning model group COM0. Therefore, compared to the technique described in Non-Patent Document 1, which generates artificial examples near the decision boundary of the support vector machine, the present exemplary embodiment provides artificial examples that are more effective in improving the accuracy of the machine learning model to be trained. can be generated.
  • generator 24 generates a plurality of artificial examples.
  • the generation unit 24 integrates two artificial examples that satisfy the similarity condition into one artificial example, among the plurality of generated artificial examples.
  • the similarity condition is a condition indicating that examples are similar.
  • the similarity condition may be, for example, that the cosine similarity is greater than or equal to a threshold, or that the distance in the feature amount space is less than or equal to the threshold.
  • similar conditions are not limited to these. Details of the process of integration will be described later.
  • FIG. 11 is a flow diagram illustrating the flow of the information processing method S20A according to the third exemplary embodiment.
  • the information processing method S20A shown in FIG. 11 is configured substantially in the same manner as the information processing method S20 according to exemplary embodiment 2, but differs in that step S205A is further included.
  • Step S205A the generation unit 24 integrates two similar artificial examples among the artificial examples generated in step S205. Specifically, the generation unit 24 determines whether or not any of the artificial example generated in step S205 this time and the artificial example generated in step S205 up to the previous time satisfies the similarity condition. If it is determined that the similarity condition is met, the generation unit 24 integrates two artificial examples that satisfy the similarity condition.
  • An example of the integration process is the process of synthesizing two artificial examples.
  • the generator 24 synthesizes two artificial examples to generate one artificial example, and deletes the original two artificial examples that satisfy the similarity condition.
  • Another example of integration processing is processing for deleting one of two artificial examples. Note that the integration process is not limited to the process described above, as long as one artificial example generated by referring to the two artificial examples is used instead of the two artificial examples that satisfy the similarity condition.
  • deleting an artificial example means deleting it from objects to be labeled in step S206 and objects to be added to training examples in step S208. As a result, the integrated artificial examples are labeled and added to the training examples.
  • the generation unit generates a plurality of artificial examples, and integrates two artificial examples satisfying similarity conditions among the plurality of generated artificial examples into one artificial example. Adopted.
  • the present exemplary embodiment generates artificial examples that can more efficiently improve the accuracy of machine learning models in areas lacking training examples by integrating artificial examples that satisfy similarity conditions. can do.
  • the generation unit 24 outputs artificial examples having variations in a plurality of prediction results obtained using the post-training machine learning model group COM0 among the generated artificial examples.
  • an artificial example with variation is an artificial example showing that the evaluation result of variation indicates "large variation”. The details of the variability evaluation have been described above and will not be repeated.
  • the generation unit 24 performs post-evaluation of variations in the generated artificial examples using the post-training machine learning model group COM0, and adopts artificial examples having variations in prediction results from the post-evaluation.
  • FIG. 12 is a flow diagram illustrating the flow of the information processing method S20B according to the fourth illustrative embodiment.
  • the information processing method S20B shown in FIG. 12 is configured substantially in the same manner as the information processing method S20 according to exemplary embodiment 2, but differs in that step S205B is further included.
  • Step S205B In step S205B, the generation unit 24 post-evaluates the artificial examples generated in step S205.
  • the generating unit 24 uses the machine learning model group COM0 to evaluate the variation in prediction results for the artificial example. For example, in the example shown in FIG. 7, the generation unit 24 evaluates the variation of the prediction results for the artificial example tv1-1 using the machine learning model group COM1.
  • the machine learning model group COM1 used to evaluate the variability is desirably the one used to evaluate the training example t1 referenced to generate the artificial example tv1-1.
  • the details of the process of evaluating the variability of prediction results using the machine learning model group COM0 are as described in the second exemplary embodiment.
  • the generation unit 24 determines that the prediction results for the artificial example generated in step S205 do not vary greatly, the generation unit 24 deletes the artificial example.
  • deleting an artificial example means deleting it from objects to be labeled in step S206 and objects to be added to training examples in step S208. As a result, artificial examples with varying prediction results are labeled and added to the training examples.
  • ⁇ Effects of this exemplary embodiment> a configuration is adopted in which the generation unit outputs artificial examples in which there are variations in a plurality of prediction results obtained using the machine learning model group after training, among the generated artificial examples. It is
  • artificial examples obtained by synthesizing multiple training examples, including training examples with varying prediction results do not necessarily have varying prediction results.
  • artificial examples generated in this way may have small variations in prediction results.
  • Training a machine learning model using training examples with small variations in prediction results is not efficient in improving the accuracy of the machine learning model.
  • the present exemplary embodiment generates artificial examples that can more efficiently improve the accuracy of machine learning models in areas lacking training examples by post-evaluating the generated artificial examples. be able to.
  • This exemplary embodiment is a form in which the configuration of the machine learning model group COM0 and steps S203 to S204 in the information processing method S20 in exemplary embodiment 2 are modified as follows. This exemplary embodiment will be described with reference to FIG. FIG. 13 is a diagram schematically illustrating an information processing method according to this exemplary embodiment.
  • Each machine learning model mj is a model generated by the same machine learning algorithm.
  • each machine learning model mj may be a decision tree.
  • Step S203 the training unit 22 extracts a training example group Dj from the training example group T acquired by the acquisition unit 21 in step S201.
  • Training example group Dj is part of training example group T.
  • the training unit 22 may extract the training example group Dj by random sampling.
  • Machine learning model groups mj1 and mj2 are trained such that the parameters constituting each are different from each other by using training example groups Dj1 and Dj2, which are at least partially different.
  • Step S204 of this exemplary embodiment the selection unit 23 evaluates the variation of prediction results for each training example included in the training example group T using the machine learning model group COM0. Further, the selection unit 23 selects training examples with variations in prediction results. In the example of FIG. 13, the selection unit 23 selects training examples t1, t3, .
  • step S205 is as described in the second exemplary embodiment. That is, in the example of FIG. 13, any one of the first generation process, the second generation process, and the third generation process is executed for each of the training examples t1, t2, . As a result, artificial examples tv1, tv2, . . . are generated.
  • This exemplary embodiment uses models generated by the same machine learning algorithm as machine learning models constituting a group of machine learning models, and selects training examples having variations in prediction results from the acquired training example group. Choose, and adopt the configuration.
  • this exemplary embodiment can generate effective artificial examples by improving the accuracy of the machine learning model when all the machine learning included in the machine learning model group is a decision tree. I will explain why. Decision trees can vary greatly in term structure for small changes in the training examples. Therefore, by using a group of machine learning models including a plurality of decision trees, it is possible to more accurately select training examples with varying prediction results.
  • Some or all of the functions of the information processing apparatuses 10 and 20 may be realized by hardware such as integrated circuits (IC chips), or may be realized by software.
  • the information processing apparatuses 10 and 20 are implemented by computers that execute instructions of programs, which are software that implements each function, for example.
  • An example of such a computer (hereinafter referred to as computer C) is shown in FIG.
  • Computer C comprises at least one processor C1 and at least one memory C2.
  • a program P for operating the computer C as the information processing apparatuses 10 and 20 is recorded in the memory C2.
  • the processor C1 reads the program P from the memory C2 and executes it, thereby realizing each function of the information processing apparatuses 10 and 20 .
  • processor C1 for example, CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit) , a microcontroller, or a combination thereof.
  • memory C2 for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.
  • the computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and temporarily storing various data.
  • Computer C may further include a communication interface for sending and receiving data to and from other devices.
  • Computer C may further include an input/output interface for connecting input/output devices such as a keyboard, mouse, display, and printer.
  • the program P can be recorded on a non-temporary tangible recording medium M that is readable by the computer C.
  • a recording medium M for example, a tape, disk, card, semiconductor memory, programmable logic circuit, or the like can be used.
  • the computer C can acquire the program P via such a recording medium M.
  • the program P can be transmitted via a transmission medium.
  • a transmission medium for example, a communication network or broadcast waves can be used.
  • Computer C can also obtain program P via such a transmission medium.
  • Appendix 1 an acquisition means for acquiring a plurality of training examples; a training means for training a machine learning model group including a plurality of machine learning models that output prediction results using examples as input, using the plurality of training examples; selection means for selecting, from among the plurality of training examples, a training example in which a plurality of prediction results obtained using the machine learning model group after training varies; generating means for generating an artificial example by synthesizing two or more training examples including the selected training example from among the plurality of training examples; Information processing device with
  • artificial examples are generated using training examples with variations in prediction results, so artificial examples are not generated in biased areas such as the vicinity of the decision boundary. Artificial examples can often be generated.
  • Appendix 2 The information processing apparatus according to appendix 1, wherein the generating means generates the artificial example by synthesizing the selected training example and an example existing in the vicinity of the selected training example in a feature space.
  • the selection means selects, from the plurality of training examples, two or more training examples in which the plurality of prediction results vary;
  • the information processing apparatus according to appendix 1, wherein the generating means generates the artificial example by synthesizing two or more of the selected training examples.
  • training examples with different prediction results are combined to generate artificial examples, so it is possible to generate artificial examples with high accuracy in regions where training examples are insufficient.
  • the generating means is a first generation process of synthesizing the selected training example and examples existing in the vicinity of the selected training example in a feature space; a second generation process that combines two or more of the selected training examples to generate the artificial example;
  • the information processing apparatus according to appendix 1, wherein the artificial example is generated by executing any one of:
  • either the first generation process or the second generation process is selectively used to generate artificial examples. Artificial examples can be generated.
  • the machine learning model group is repeatedly trained using the training examples to which the generated artificial examples are added, so it is possible to more accurately select training examples with varying prediction results.
  • artificial examples can be generated in areas where artificial examples are scarce.
  • the generating means is generating a plurality of said artificial examples; 5.
  • the information processing apparatus according to any one of appendices 1 to 4, wherein two artificial examples satisfying a similarity condition among the plurality of artificial examples are integrated into one artificial example.
  • Appendix 7 The method according to any one of appendices 1 to 6, wherein the generating means outputs, among the artificial examples, artificial examples in which a plurality of prediction results obtained using the group of machine learning models after training vary. Information processing equipment.
  • Appendix 8 The information processing apparatus according to any one of appendices 1 to 7, wherein the machine learning model group includes a training target machine learning model trained using the artificial example.
  • Appendix 9 The information processing apparatus according to any one of appendices 1 to 8, wherein at least two of the machine learning model groups use different machine learning algorithms.
  • Appendix 11 11. The information processing device according to any one of appendices 1 to 10, wherein at least one of the machine learning model groups is a decision tree.
  • Appendix 12 12. The information processing apparatus according to any one of appendices 1 to 11, further comprising labeling means for labeling some or all of the plurality of training examples and the artificial examples.
  • (Appendix 13) obtaining multiple training examples; training, using the plurality of training examples, a machine learning model group including a plurality of machine learning models that output prediction results using examples as input; Selecting, from among the plurality of training examples, a training example in which a plurality of prediction results obtained using the machine learning model group after training varies; synthesizing two or more training examples including the selected training example from among the plurality of training examples to generate an artificial example; Information processing method including.
  • a program for causing a computer to function as an information processing device comprising: an acquisition means for acquiring a plurality of training examples; a training means for training a machine learning model group including a plurality of machine learning models that output prediction results using examples as input, using the plurality of training examples; selection means for selecting, from among the plurality of training examples, a training example in which a plurality of prediction results obtained using the machine learning model group after training varies; generating means for generating an artificial example by synthesizing two or more training examples including the selected training example from among the plurality of training examples; A program that acts as a
  • Appendix 15 A computer-readable recording medium on which the program according to appendix 14 is recorded.
  • said processor comprising: an acquisition process for acquiring a plurality of training examples; a training process of training a machine learning model group including a plurality of machine learning models that output prediction results using examples as input, using the plurality of training examples; A selection process of selecting, from among the plurality of training examples, a training example in which a plurality of prediction results obtained using the machine learning model group after training varies; and a generation process of generating an artificial example by synthesizing two or more training examples including the selected training example from among the plurality of training examples.
  • the information processing apparatus may further include a memory, and the memory stores a program for causing the processor to execute the -process, the -process, and the -process. good too.
  • this program may be recorded in a computer-readable non-temporary tangible recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2021/020174 2021-05-27 2021-05-27 情報処理装置、情報処理方法、及びプログラム Ceased WO2022249392A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2021/020174 WO2022249392A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法、及びプログラム
US18/561,354 US20240242123A1 (en) 2021-05-27 2021-05-27 Information processing apparatus, information processing method, and storage medium
JP2023523860A JP7571879B2 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法、及びプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/020174 WO2022249392A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法、及びプログラム

Publications (1)

Publication Number Publication Date
WO2022249392A1 true WO2022249392A1 (ja) 2022-12-01

Family

ID=84229594

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/020174 Ceased WO2022249392A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法、及びプログラム

Country Status (3)

Country Link
US (1) US20240242123A1 (https=)
JP (1) JP7571879B2 (https=)
WO (1) WO2022249392A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12353509B1 (en) * 2020-12-16 2025-07-08 Amdocs Development Limited System, method, and computer program for identifying a hidden class

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019022136A1 (ja) * 2017-07-25 2019-01-31 国立大学法人東京大学 学習方法、学習プログラム、学習装置及び学習システム
US20190080205A1 (en) * 2016-09-16 2019-03-14 Deep Learning Analytics, LLC Systems and Methods for Deep Model Translation Generation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190080205A1 (en) * 2016-09-16 2019-03-14 Deep Learning Analytics, LLC Systems and Methods for Deep Model Translation Generation
WO2019022136A1 (ja) * 2017-07-25 2019-01-31 国立大学法人東京大学 学習方法、学習プログラム、学習装置及び学習システム

Also Published As

Publication number Publication date
JP7571879B2 (ja) 2024-10-23
JPWO2022249392A1 (https=) 2022-12-01
US20240242123A1 (en) 2024-07-18

Similar Documents

Publication Publication Date Title
US20190354896A1 (en) Information processing apparatus, information processing method, and computer-readable storage medium
CN113449587B (zh) 一种人体行为识别和身份认证方法、装置和电子设备
WO2021086701A1 (en) Protecting deep learned models
CN113179660A (zh) 基于用于深度神经网络的增强学习的张量分解中的排序选择
CN114118384B (zh) 神经网络模型的量化方法、可读介质和电子设备
JP6232522B2 (ja) 計算機及びグラフデータ生成方法
CN115982596A (zh) 用于多模态数据处理的方法、装置、设备和介质
JP2024080175A (ja) 質問生成装置、質問生成方法、およびプログラム
CN115335831A (zh) 用于训练稀疏神经网络同时保持稀疏性的系统和方法
JP2024502154A (ja) 訓練装置、分類装置、訓練方法、分類方法、及びプログラム
JP7559943B2 (ja) 情報処理装置、情報処理方法及びプログラム
CN110069284A (zh) 一种基于opu指令集的编译方法及编译器
WO2022003991A1 (ja) ２次元マップ生成装置、２次元マップ生成方法および２次元マップ生成用プログラム
JPWO2022249373A5 (https=)
JP7571879B2 (ja) 情報処理装置、情報処理方法、及びプログラム
JPWO2022249392A5 (https=)
CN111385601A (zh) 一种视频审核的方法及系统
JP7670156B2 (ja) 発明の名称 ： 情報処理装置、情報処理方法、及び、プログラム
De Luca et al. Towards a GPU parallel software for environmental data fitting
CN113140222B (zh) 一种声纹向量提取方法、装置、设备及存储介质
CN118038857A (zh) 训练自动语音识别器的训练示例的生成
JP2022151502A (ja) プログラム、情報処理装置、及び方法
CN110188773B (zh) 图像处理方法及装置
JP2018195089A (ja) クラスタリング方法、クラスタリングプログラム、および情報処理装置
JP7003435B2 (ja) 情報処理装置、プログラム、情報処理方法及びデータ構造

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943041

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18561354

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2023523860

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21943041

Country of ref document: EP

Kind code of ref document: A1