WO2022249373A1 - 情報処理装置、情報処理方法及びプログラム - Google Patents

情報処理装置、情報処理方法及びプログラム Download PDF

Info

Publication number
WO2022249373A1
WO2022249373A1 PCT/JP2021/020115 JP2021020115W WO2022249373A1 WO 2022249373 A1 WO2022249373 A1 WO 2022249373A1 JP 2021020115 W JP2021020115 W JP 2021020115W WO 2022249373 A1 WO2022249373 A1 WO 2022249373A1
Authority
WO
WIPO (PCT)
Prior art keywords
training
examples
machine learning
training examples
artificial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/020115
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
優太 畠山
穣 岡嶋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to PCT/JP2021/020115 priority Critical patent/WO2022249373A1/ja
Priority to JP2023523844A priority patent/JP7559943B2/ja
Priority to US18/561,357 priority patent/US20240249205A1/en
Publication of WO2022249373A1 publication Critical patent/WO2022249373A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Definitions

  • the present invention relates to technology for generating examples used in machine learning.
  • Non-Patent Document 1 the minority class instance (training example) closest to the decision boundary of the Support Vector Machine and the neighboring minority class instance are synthesized to create a minority class virtual instance. It is described to generate
  • the virtual instance generated by the technique described in Non-Patent Document 1 may be generated at a location farther from the decision boundary than the instance of the minority class closest to the decision boundary. Artificial examples generated at such locations do not necessarily improve the estimation accuracy of support vector machines efficiently. Thus, the artificial examples generated by the technique described in Non-Patent Document 1 have room for improvement in terms of efficiently improving the estimation accuracy of the machine learning model.
  • One aspect of the present invention has been made in view of the above problems, and an example of its purpose is to provide a technique for generating artificial examples that more efficiently improve the prediction accuracy of a machine learning model. .
  • An information processing apparatus uses an acquisition unit that acquires a plurality of training examples, and one or more machine learning models that output a prediction result from an input of the training examples among the plurality of training examples. selection means for selecting two or more training examples for which one or more obtained prediction results are uncertain; and generating means for generating an artificial example by synthesizing the two or more training examples selected by the selection means. .
  • An information processing method is characterized in that an information processing apparatus obtains a plurality of training examples, and one or more machine learning models that output prediction results using the training examples as input from among the plurality of training examples. selecting two or more training examples for which one or more of the prediction results obtained using is uncertain, and combining the selected two or more training examples to generate an artificial example.
  • a program according to one aspect of the present invention is a program for causing a computer to function as an information processing device, the computer comprising: an acquisition unit configured to acquire a plurality of training examples; selection means for selecting two or more training examples with uncertain one or more prediction results obtained using one or more machine learning models that output prediction results as inputs; and two or more selected by the selection means. and generating means for synthesizing training examples to generate artificial examples.
  • FIG. 1 is a block diagram showing the configuration of an information processing device according to exemplary Embodiment 1 of the present invention
  • FIG. FIG. 3 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 1 of the present invention
  • FIG. 4 is a diagram schematically showing a specific example of an information processing method according to exemplary embodiment 1 of the present invention
  • FIG. 7 is a block diagram showing the configuration of an information processing apparatus according to exemplary Embodiment 2 of the present invention
  • FIG. 7 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 2 of the present invention
  • FIG. 10 is a diagram schematically showing a specific example of first selection processing according to exemplary embodiment 2 of the present invention
  • FIG. 10 is a diagram schematically showing a specific example of second selection processing according to exemplary embodiment 2 of the present invention
  • FIG. 11 is a flow diagram showing the flow of generation processing according to exemplary embodiment 3 of the present invention
  • FIG. 11 is a flow diagram showing the flow of a first generation process according to exemplary embodiment 3 of the present invention
  • FIG. 12 is a flow diagram showing the flow of a second generation process according to exemplary embodiment 3 of the present invention
  • FIG. 12 is a diagram schematically showing a specific example of an information processing method according to illustrative embodiment 4 of the present invention
  • FIG. 11 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 5 of the present invention
  • FIG. 11 is a flow diagram illustrating the flow of an information processing method according to exemplary embodiment 6 of the present invention.
  • 1 is a block diagram showing the configuration of a computer functioning as an information processing device according to illustrative embodiments 1 to 6 of the present invention;
  • FIG. 11 is a flow diagram illustrating the flow of an information processing method according to exemplary embodiment 6 of the present invention.
  • 1 is a block diagram showing the configuration of a computer functioning as an information processing device according to illustrative embodiments 1 to 6 of the present invention;
  • FIG. 1 is a block diagram showing the configuration of an information processing device 10. As shown in FIG.
  • the information processing device 10 is a device that generates an artificial example for use in training a machine learning model from a plurality of examples.
  • the information processing device 10 includes an acquisition unit 11, a selection unit 12, and a generation unit 13, as shown in FIG.
  • the acquisition unit 11 is an example of a configuration that implements the acquisition means described in the claims.
  • the selection unit 12 is an example of a configuration that implements the selection means described in the claims.
  • the generation unit 13 is an example of a configuration that implements the generation means described in the claims.
  • the acquisition unit 11 acquires a plurality of training examples.
  • the selecting unit 12 selects one or more uncertain training examples obtained by using one or more machine learning models that output prediction results from the training examples obtained by the obtaining unit 11.
  • Select two or more of The generator 13 synthesizes two or more training examples selected by the selector 12 to generate an artificial example.
  • An example is information that is input to a machine learning model and includes features. In other words, examples exist in feature space.
  • a training example is an example that can be used for training each of one or more machine learning models. The training examples may be examples obtained by observation or may be artificial examples that are artificially generated.
  • Each of the one or more machine learning models takes an example as input and outputs a prediction result.
  • a prediction result may include, for example, a predicted probability that each of a plurality of labels is predicted. In this case, the label with the highest prediction probability may be referred to as the prediction result.
  • Each of the one or more machine learning models is, by way of example, a model generated using a machine learning algorithm such as a decision tree, neural network, random forest, or support vector machine. However, the machine learning algorithms used to generate each machine learning model are not limited to these.
  • One or more machine learning models may be stored, for example, in the memory of the information processing device 10 or may be stored in another device communicably connected to the information processing device 10 .
  • Some or all of the one or more machine learning models may be models trained using some or all of the training examples acquired by the acquisition unit 11 . Also, part or all of the one or more machine learning models may be models trained using training examples other than the training examples acquired by the acquisition unit 11 .
  • the one or more machine learning models do not necessarily have to be "machine learning models to be trained using generated artificial examples".
  • the one or more machine learning models may include some or all of the machine learning models being trained.
  • the one or more machine learning models may not include a machine learning model to be trained.
  • the number of machine learning models to be trained may be plural or singular.
  • Training examples with uncertain prediction results are training examples with low confidence in the prediction results of one or more machine learning models.
  • a training example with an uncertain prediction result is, for example, a training example with an uncertainty evaluation result that satisfies a predetermined condition.
  • a training example with uncertain prediction results is, for example, a training example in which a plurality of prediction results obtained using a plurality of machine learning models vary.
  • evaluating uncertainty means evaluating variations in a plurality of prediction results, for example, evaluating whether the variations are large.
  • a training example in which multiple prediction results vary is a training example that indicates that the variation evaluation result is "large variation.”
  • evaluation of variability means evaluating whether or not the variability of a plurality of prediction results is large.
  • the variability evaluation may be an evaluation based on the entropy of voting results. The entropy of voting results will be described in detail in exemplary embodiment 2 below.
  • the evaluation of variation may be an evaluation based on the ratio of prediction results showing the same label among a plurality of prediction results.
  • the evaluation of variation is not limited to the one described above.
  • "a training example evaluated as having a large variation in a plurality of prediction results” is also referred to as a "training example with a variation in prediction results”.
  • a training example evaluated as not having a large variation in a plurality of prediction results is also described as a "training example having a small variation in prediction results”.
  • a training example with an uncertain prediction result is, for example, a training example that exists near the decision boundary in the feature space of at least one machine learning model.
  • evaluating the uncertainty means evaluating whether or not the training example exists in the vicinity of the decision boundary, and the predetermined condition is the condition that it exists in the vicinity of the decision boundary.
  • FIG. 2 is a flow diagram showing the flow of the information processing method S10.
  • step S101 acquisition process
  • the acquisition unit 11 acquires a plurality of training examples.
  • the acquisition unit 11 may acquire a plurality of training examples by reading them from memory.
  • the acquisition unit 11 may acquire a plurality of training examples from an input device or from a device connected via a network.
  • the plurality of training examples obtained in this step includes one or both of observed examples and artificial examples.
  • step S102 selection processing
  • the selection unit 12 selects one or more prediction results obtained using one or more machine learning models that output prediction results from training examples as input, and one or more prediction results are uncertain. Select two or more training examples.
  • step S103 generation processing
  • the generation unit 13 generates an artificial example by synthesizing the two or more training examples selected in step S102.
  • the generator 13 may synthesize two training examples to generate one artificial example, or may synthesize three or more training examples to generate one artificial example. Further, the generation unit 13 may generate one artificial example, or may generate a plurality of artificial examples.
  • the generation unit 13 generates an artificial example using the following equation (1), for example.
  • Equation (1) ⁇ xv represents an artificial example, and x i and x j represent training examples selected by the selection unit 12 .
  • ⁇ is a weighting coefficient that satisfies 0 ⁇ 1.
  • the generation unit 13 determines the value of the coefficient ⁇ using random numbers generated by a random function. Note that the generation processing performed by the generation unit 13 is not limited to the method described above, and the generation unit 13 may synthesize a plurality of training examples by other methods.
  • FIG. 3 is a diagram schematically showing a specific example of the information processing method S10.
  • the training example group T acquired by the acquisition unit 11 in step S101 includes training examples t1, t2, t3, .
  • Each of the plurality of machine learning models mj is a model trained to output a label indicating "A" or "B" as a prediction result when an example is input.
  • the selection unit 12 inputs a plurality of training examples t1 to t10 to be evaluated to each of a plurality of machine learning models mj. As a result, the selection unit 12 selects a plurality of prediction results including a prediction result output from the machine learning model m1, a prediction result output from the machine learning model m2, a prediction result output from the machine learning model m3, and so on. get Further, the selection unit 12 selects training examples t3, t4, t7, and t8 with uncertain prediction results from the plurality of training examples t1 to t10 based on the plurality of prediction results.
  • the generation unit 13 generates a training example t51 by synthesizing the training example t3 and the training example t4 selected by the selection unit 12 .
  • the generation unit 13 also generates a training example t52 by synthesizing the training example t3 and the training example t8 selected by the selection unit 12 .
  • the generation unit 13 also generates a training example t53 by synthesizing the training example t7 and the training example t8 selected by the selection unit 12 .
  • ⁇ Effects of this exemplary embodiment> As described above, in the information processing apparatus 10 according to the present exemplary embodiment, one or a plurality of machine learning models obtained using one or a plurality of machine learning models that output prediction results from training examples as inputs. Two or more training examples whose prediction results are uncertain are selected, and the selected two or more training examples whose prediction results are uncertain are combined to generate an artificial example. Artificial examples obtained by synthesizing two or more training examples with uncertain prediction results are likely to be generated where prediction results are scarce. Therefore, by training a machine learning model using the generated artificial examples, it is possible to more efficiently improve the prediction accuracy of the machine learning model to be trained.
  • FIG. 4 is a block diagram showing the configuration of the information processing device 20.
  • the information processing device 20 is a device that generates artificial examples used for training a machine learning model.
  • the information processing device 20 includes an acquisition unit 21 , a training unit 22 , a selection unit 23 , a generation unit 24 , a labeling unit 25 , an output unit 26 and a control unit 27 .
  • the training unit 22 is an example of a configuration that implements training means described in the claims.
  • the labeling unit 25 is an example of a configuration that implements the labeling means described in the claims.
  • the output unit 26 is an example of a configuration that implements the output means described in the claims.
  • the acquisition unit 21 is configured in the same manner as the acquisition unit 11 in the first exemplary embodiment.
  • the training unit 22 uses some or all of the plurality of training examples acquired by the acquisition unit 21 to train some or all of the one or more machine learning models.
  • a plurality of machine learning models is also referred to as a "machine learning model group”.
  • One or more machine learning models include, for example, a machine learning model to be trained using artificial examples generated by the information processing device 20 . Also, at least one of the one or more machine learning models may be a decision tree, as an example.
  • the selection unit 23 selects one or more training examples with uncertain prediction results obtained using one or more machine learning models that output prediction results from the training examples obtained by the acquisition unit 21. Select two or more of Selection processing performed by the selection unit will be described later.
  • the generation unit 24 generates an artificial example by synthesizing two or more training examples selected by the selection unit 23 .
  • the generation unit 24 generates an artificial example by performing the synthesis process represented by the above formula (1).
  • the labeling unit 25 labels some or all of the training examples and artificial examples.
  • the label assigning unit 25 may assign a label based on information output from an input device that accepts user operations. Further, as an example, the labeling unit 25 may give labels obtained by inputting training examples and artificial examples to a machine learning model trained to output labels with examples as input.
  • the machine learning model that outputs the label is, for example, at least one machine learning model or a model with higher prediction accuracy than the machine learning model to be trained. If the machine learning model to be trained is a decision tree, the machine learning model that outputs labels is, for example, a random forest.
  • the output unit 26 outputs the artificial example generated by the generation unit 24.
  • the output unit 26 may store the artificial example generated by the generation unit 24 in a recording medium such as an external storage device.
  • the output unit 26 may output the artificial example to an output device such as a display device, for example.
  • the control unit 27 controls each unit of the information processing device 20 .
  • the control unit 27 specifically adds the artificial examples generated by the generation unit 24 to a plurality of training examples, and causes the acquisition unit 21, the training unit 22, the selection unit 23, and the generation unit 24 to function again. .
  • FIG. 5 is a flow diagram showing the flow of the information processing method S20.
  • Step S201 the acquisition unit 21 acquires a plurality of training examples.
  • the plurality of training examples to be acquired may include examples obtained by observation, or may include artificial examples.
  • Step S202 In step S ⁇ b>202 , the label assigning unit 25 assigns a label to each of the plurality of training examples acquired by the acquiring unit 21 .
  • step S203 the training unit 22 trains one or more machine learning models using some or all of the training examples acquired by the acquisition unit 21 .
  • the training unit 22 trains one or more machine learning models using the training example group Dj.
  • a set of training examples Dj is a set of training examples used for training the machine learning model.
  • the training example group is a set of training examples randomly extracted by the training unit 22 from the plurality of training examples acquired by the acquisition unit 21 .
  • the set of training examples used to train one machine learning model may overlap, in part or all, with the set of training examples used to train other machine learning models.
  • At least two of the multiple machine learning models may use different machine learning algorithms.
  • each of the plurality of machine learning models may use the same machine learning algorithm.
  • step S204 the selection unit 23 selects two or more training examples for which one or more prediction results obtained using one or more machine learning models are uncertain from among the plurality of training examples acquired by the acquisition unit 21. do.
  • the selection processing performed by the selection unit 23 will be described later.
  • the training examples selected by the selection unit 23 are also referred to as “uncertain training examples”.
  • Step S205 the generator 24 synthesizes two or more training examples selected by the selector 23 to generate an artificial example.
  • the generation unit 24 generates an artificial example by performing the synthesis process represented by the above formula (1).
  • the generation unit 24 may use known techniques such as MUNGE (see reference 1) and SMOTE (see reference 2) as a technique for synthesizing a plurality of training examples.
  • the generation unit 24 When synthesizing a portion of the training examples selected by the selection unit 23, the generation unit 24 identifies a portion of the training examples selected by the selection unit 23 as synthesis targets, and synthesizes the identified training examples to generate artificial examples. Generate.
  • the number of training examples that the generator 24 identifies as synthesis targets may be two, or three or more.
  • the generation unit 24 may randomly specify a plurality of training examples as objects to be synthesized from among the plurality of training examples selected by the selection unit 23, and the distance in the feature amount space is equal to or less than a threshold. Certain training examples may be specified. Note that the method of specifying training examples to be synthesized by the generation unit 24 is not limited to these.
  • step S206 the label assigning unit 25 assigns labels to the one or more artificial examples generated by the generating unit 24, respectively.
  • step S207 the control unit 27 determines whether or not to end the training process. As an example, when the number of times the processes of steps S203 to S206 have been executed is equal to or greater than a predetermined threshold, the control unit 27 determines to end the training process. On the other hand, when the number of times the processes of steps S203 to S206 have been executed is less than the predetermined threshold, it is determined that the training process is not finished. If the training process is not to end (NO in step S207), the control unit 27 proceeds to the process of step S208. On the other hand, when ending the training process (YES in step S207), the control unit 27 proceeds to the process of step S209.
  • Step S208 the control unit 27 adds one or more labeled artificial examples to a plurality of training examples. After completing the process of step S208, the control unit 27 returns to the process of step S203. In other words, the control unit 27 adds the artificial example to the plurality of training examples and causes the acquisition unit 21, the training unit 22, the selection unit 23, and the generation unit to function again.
  • step S209 the output unit 26 outputs the artificial example generated by the generation unit 24 .
  • the output unit 26 outputs one or more unreliable prediction results obtained using one or more machine learning models trained by the training unit 22 from among the artificial examples generated by the generation unit 24. Output one or more.
  • the first selection process to the third selection process will be described.
  • the two or more training examples selected by the selection unit 23 include, for example, training examples in which prediction results obtained using a plurality of machine learning models vary.
  • the two or more training examples selected by the selection unit 23 include, as an example, training examples existing near the decision boundary in the feature space of at least one machine learning model.
  • the selection unit 23 selects training examples using predicted probabilities of at least one machine learning model.
  • the selection unit 23 selects a training example to be synthesized by executing at least one of the first selection process to the third selection process. Note that the selection process performed by the selection unit 23 is not limited to these, and the selection unit 23 may select training examples with uncertain prediction results by other methods.
  • the first selection process is a process that can be executed using multiple machine learning models.
  • the selection unit 23 selects training examples in which a plurality of prediction results obtained using a plurality of machine learning models vary.
  • FIG. 6 is a diagram schematically showing a specific example of the first selection process.
  • a training example group T includes a plurality of training examples t1, t2, t3, .
  • the machine learning model group COM includes a plurality of machine learning models m1, m2, .
  • the machine learning model m1 is a model trained by the training unit 22 using the training example group D1 included in the training example group T.
  • the machine learning model m2 is a model trained by the training unit 22 using the training example group D2 included in the training example group T.
  • the selection unit 23 uses a plurality of machine learning models mj trained by the training unit 22 to select training examples to be synthesized.
  • the selection unit 23 inputs a plurality of training examples to be evaluated from the training example group T acquired by the acquisition unit 21 to each of the plurality of machine learning models mj.
  • the selection unit 23 obtains a plurality of prediction results including a prediction result output from the machine learning model m1, a prediction result output from the machine learning model m2, and so on. Further, the selection unit 23 selects training examples t1, t2, .
  • the training examples that the selection unit 23 inputs to the plurality of machine learning models mj, that is, the training examples to be evaluated are, for example, those among the training example group T that the training unit 22 does not use for training any of the machine learning models mj. This is a training example.
  • the selection unit 23 selects a training example with variations in prediction results, using an index of vote entropy in the QBC (query by committee) method as an example.
  • equation (2) indicates to select the training example ⁇ x with the largest entropy of the voting result.
  • C indicates the total number of machine learning models.
  • V(y) indicates the number of machine learning models that predicted label y.
  • the selection unit 23 may select ⁇ x indicated by the equation (2) as a training example in which prediction results vary. Further, the selection unit 23 may select a predetermined number of training examples t1, t2, . . . in descending order of entropy, or may select training examples t1, t2, .
  • the prediction results when training examples t1 to t3 to be evaluated are input to machine learning models m1 to m2, respectively, are as follows.
  • the prediction result is "A” when the training example t1 is input to the machine learning model m1, and the prediction result is "B” when the training example t1 is input to the machine learning model m2.
  • the prediction result is "A” when the training example t2 is input to the machine learning model m1, and the prediction result is "B” when the training example t2 is input to the machine learning model m2.
  • the prediction result is "A” when the training example t3 is input to the machine learning model m1, and the prediction result is "A” when the training example t3 is input to the machine learning model m2.
  • the selection unit 23 selects training examples t1 and t2 with the maximum entropy as training examples with uncertain prediction results.
  • the generation unit 24 synthesizes the training example t1 and the training example t2 selected by the selection unit 23 to generate the artificial example tv1.
  • the second selection process is a process executable using at least one machine learning model when the machine learning model is a support vector machine.
  • the selection unit 23 selects training examples existing near the decision boundary in the feature amount space of the machine learning model.
  • FIG. 7 is a diagram schematically showing a specific example of the second selection process.
  • the selection unit 23 selects a plurality of training examples existing near the decision boundary B indicated by the machine learning model as training examples with uncertain prediction results.
  • the selection unit 23 may select training examples whose distance from the decision boundary B is equal to or less than a predetermined threshold. You may choose.
  • the selection unit 23 selects five training examples t23 to t27 in descending order of distance from the decision boundary B from among the plurality of training examples t21 to t29.
  • the selection unit 23 may select, from among the plurality of training examples, a plurality of training examples included in one of a plurality of spaces separated by the decision boundary B in the feature space. Further, the selection unit 23 may select, from among the plurality of training examples, training examples included in each of a plurality of spaces separated by the decision boundary B in the feature amount space. In other words, the selection unit 23 may select a plurality of training examples predicted with the same label, or may select training examples predicted with different labels.
  • the generation unit 24 synthesizes training examples t23 and t24 included in space R2 of spaces R1 and R2 partitioned by decision boundary B to generate artificial example t121.
  • the generator 24 also synthesizes the training example t24 contained in the space R2 and the training example t26 contained in the space R1 to generate the artificial example t122.
  • the generation unit 24 also synthesizes the training example t25 included in the space R2 and the training example t27 included in the space R1 to generate the artificial example t123.
  • an artificial example t122 obtained by synthesizing the training examples included in the spaces R1 and R2 is generated closer to the decision boundary B than the training examples t23 and t24 used for synthesis.
  • an artificial example t123 obtained by synthesizing the training examples included in the spaces R1 and R2 is generated closer to the decision boundary B than the training examples t25 and t27 used for synthesis.
  • a third selection process is a process that can be performed using at least one machine learning model.
  • the selection unit 23 selects a training example using the predicted probability of each label output from the machine learning model.
  • the selection unit 23 selects an uncertain training example ⁇ x according to the following equation (3) or (4).
  • Formula (3) is a formula that expresses the so-called Least Confident method of selecting a training example ⁇ x with the minimum prediction probability maxP(y
  • Equation (4) is the minimum difference between the prediction probability P(y1
  • This formula represents a so-called Margin Sampling technique for selecting a training example ⁇ x.
  • one or more machine learning models are trained using some or all of the plurality of training examples obtained by the obtaining unit 21, and one or more trained machines are trained.
  • the learning model is used to select training examples for synthesis.
  • the machine learning model to be trained is a decision tree
  • the structure of the decision tree can be greatly changed by slightly changing the training examples. Therefore, the prediction accuracy of decision trees is lower than that of other complex machine learning models.
  • the prediction accuracy of the machine learning model to be trained can be improved. can be substantially improved.
  • the information processing apparatus 20 selects training examples in which a plurality of prediction results obtained using a plurality of machine learning models vary in the first selection process.
  • the prediction accuracy of the machine learning model to be trained can be more effectively improved.
  • the artificial example t122 is generated at a position closer to the decision boundary B than the training examples t23 and t24 used for synthesis.
  • the artificial example t123 is generated at a position closer to the decision boundary B than the training examples t25 and t27 used for synthesis.
  • the information processing apparatus 20 synthesizes training examples respectively included in a plurality of spaces delimited by the decision boundary B, thereby generating an artificial example at a position closer to the decision boundary B than the training example used for synthesis. be able to.
  • the prediction accuracy of the machine learning model to be trained can be improved more efficiently.
  • the information processing apparatus 20 performs not only the process of selecting training examples existing near the decision boundary of the support vector machine (the second selection process described above), but also other selection processes. Select artificial examples using the uncertain training examples selected by the process (first selection process, third selection process, etc. above). As a result, even if the machine learning model to be trained is not a support vector machine, it is possible to efficiently improve the prediction accuracy of a machine learning model different from a support vector machine.
  • the information processing apparatus 20 selects uncertain training examples using a group of machine learning models including a plurality of machine learning models, so that artificial You can generate examples. In other words, it is possible to prevent excessive generation of artificial examples biased near the decision boundary of the support vector machine.
  • the information processing device 20 trains one or more machine learning models using the generated artificial examples.
  • the information processing device 20 trains one or more machine learning models using the generated artificial examples.
  • the selection unit 23 selects a training example with an uncertain prediction result by at least one of the first selection process to the third selection process.
  • the method to be selected is not limited to the methods exemplified in the exemplary embodiments described above.
  • the selection unit 23 may select training examples with uncertain prediction results by other methods.
  • the selection unit 23 may use an index of cosensus entropy to select training examples with uncertain prediction results.
  • the generation unit 24 of the information processing apparatus 20 generates a plurality of artificial examples by repeatedly executing an artificial example generation process for generating artificial examples.
  • the artificial example generation process selects either the first generation process or the second generation process based on a predetermined condition, and executes the selected process to generate an artificial example. This is the process to generate.
  • the first generation process is a process of synthesizing a plurality of training examples selected by the selection unit 23 .
  • the second generation process at least one of the plurality of training examples selected by the selection unit 23 is extracted, and the extracted training example and the training examples existing in the vicinity of the extracted training example in the feature amount space are generated. This is a process of synthesizing the
  • FIG. 8 is a flow diagram showing the flow of the artificial example generation process S30 executed by the generation unit 24 according to this exemplary embodiment.
  • the generation unit 24 performs the processing of steps S301 to S304 for each of the uncertain training examples selected by the selection unit 23 .
  • step S301 the generation unit 24 selects either the first generation process or the second generation process based on a predetermined condition.
  • the generator 24 selects either the first generation process or the second generation process based on a probability calculated based on random numbers generated by a random function.
  • step S302 the generation unit 24 determines which one is selected. When the first generation process is selected, the generation unit 24 proceeds to the process of step S303 and executes the first generation process. On the other hand, when the second generation process is selected, the generation unit 24 proceeds to the process of step S304 and executes the second generation process.
  • FIG. 9 is a flow diagram showing the flow of the first generation processing S40 performed by the generation unit 24.
  • the generation unit 24 identifies a plurality of training examples that are part of the plurality of training examples selected by the selection unit 23 .
  • This specifying process is the same as the specifying process performed by the selection unit 23 in the second exemplary embodiment described above.
  • FIG. 10 is a flow diagram showing the flow of the second generation processing S50 performed by the generation unit 24.
  • step S ⁇ b>501 the generation unit 24 identifies the closest training example to the training example whose prediction result is uncertain in the feature amount space.
  • step S502 the generation unit 24 synthesizes the uncertain training example and the nearest training example identified in step S501 to generate an artificial example.
  • the information processing apparatus 20 selects the first generation process and the second generation process based on a predetermined condition, and executes the selected process to generate an artificial example. repeatedly.
  • the information processing apparatus 20 can generate artificial examples having more diverse features. In other words, the information processing apparatus 20 can prevent the generated artificial examples from becoming uniform.
  • FIG. 11 is a diagram schematically showing a specific example of the information processing method according to this exemplary embodiment.
  • the machine learning model group COM0 includes a plurality of machine learning model groups COM1, COM2, .
  • the selection unit 23 of the information processing device 20 selects uncertain training examples using a plurality of machine learning model groups COMi (1 ⁇ i ⁇ M; M is an integer equal to or greater than 2).
  • the machine learning model group COM1 includes machine learning models m1-1, m1-2, .
  • the machine learning model group COM2 includes machine learning models m2-1, m2-2, .
  • the training unit 22 extracts the training example group Di from the training example group T acquired by the acquisition unit 21 .
  • the training unit 22 extracts a training example group Di from the training example group T by random sampling.
  • the training unit 22 trains the machine learning model group COMi using the extracted training example group Di. That is, the training unit 22 trains the machine learning models m1-1, m1-2, . . . using the training example group D1. The training unit 22 also trains the machine learning models m2-1, m2-2, . . . using the training example group D2.
  • the training unit 22 uses the machine learning model group COMi to calculate information representing the uncertainty of training examples not used for training.
  • An example of information representing the uncertainty of the training examples is the entropy in equation (2) above.
  • the training unit 22 calculates information indicating uncertainty based on prediction results obtained by inputting training examples that were not used for training to each machine learning model mi-j included in the machine learning model group COMi. do.
  • the training unit 22 assigns training examples not used in the training of the machine learning model group COM1 (that is, training examples not included in the training example group D1) to the machine learning models m1-1, m1-2, . to get the prediction results.
  • the training unit 22 calculates information indicating uncertainty for each of the plurality of training examples input to the machine learning model group COM1, based on the obtained plurality of prediction results.
  • the training unit 22 inputs training examples not used in the training of the machine learning model group COM2 (that is, training examples not included in the training example group D2) to each of the machine learning models m2-1, m2-2, . to get the prediction results.
  • the training unit 22 calculates information indicating uncertainty for each of the plurality of training examples input to the machine learning model group COM2, based on the obtained plurality of prediction results.
  • the selection unit 23 selects training examples t1-1, t1-2, . to select. Further, the selection unit 23 selects training examples t2-1, t2-2, .
  • the generating unit 24 selects two or more training examples from among the training examples t1-1, t1-2, . . . , the training examples t2-1, t2-2, . Synthesize to generate artificial example tv1-1.
  • the generator 24 synthesizes the training example t1-1 and the training example t1-2 selected by the selector 23 to generate the artificial example tv1-1.
  • the generation unit 24 synthesizes the training example t2-1 selected by the selection unit 23 and the training example t2-2 to generate an artificial example tv2-1.
  • the combination of training examples to be combined is not limited to the combination shown in FIG. 11, and may be another combination.
  • the exemplary embodiment employs a configuration in which machine learning models are used to select training examples with uncertain prediction results.
  • this exemplary embodiment suppresses the generation of artificial examples in biased regions compared to the case of generating artificial examples in the vicinity of the decision boundary like the technique described in Non-Patent Document 1. be able to.
  • generator 24 generates a plurality of artificial examples.
  • the generating unit 24 integrates two artificial examples satisfying the similarity condition among the plurality of artificial examples into one artificial example and outputs the same.
  • the similarity condition is a condition indicating that examples are similar.
  • the similarity condition may be, for example, that the cosine similarity is greater than or equal to a threshold, or that the distance in the feature amount space is less than or equal to the threshold.
  • similar conditions are not limited to these. Details of the process of integration will be described later.
  • FIG. 12 is a flow diagram illustrating the flow of the information processing method S20A according to the fifth exemplary embodiment.
  • the information processing method S20A shown in FIG. 12 is configured substantially in the same manner as the information processing method S20 according to exemplary embodiment 2, but differs in that step S205A is further included.
  • Step S205A the generation unit 24 integrates two similar artificial examples among the artificial examples generated in step S205. Specifically, the generation unit 24 determines whether or not any of the artificial example generated in step S205 this time and the artificial example generated in step S205 up to the previous time satisfies the similarity condition. If it is determined that the similarity condition is met, the generation unit 24 integrates two artificial examples that satisfy the similarity condition.
  • An example of the integration process is the process of synthesizing two artificial examples.
  • the generator 24 synthesizes two artificial examples to generate one artificial example, and deletes the original two artificial examples that satisfy the similarity condition.
  • Another example of integration processing is processing for deleting one of two artificial examples. Note that the integration process is not limited to the process described above, as long as one artificial example generated by referring to the two artificial examples is used instead of the two artificial examples that satisfy the similarity condition.
  • deleting an artificial example means deleting it from objects to be labeled in step S206 and objects to be added to training examples in step S208. As a result, the integrated artificial examples are labeled and added to the training examples.
  • the generation unit generates a plurality of artificial examples, and integrates two artificial examples satisfying similarity conditions among the plurality of generated artificial examples into one artificial example. Adopted.
  • the present exemplary embodiment generates artificial examples that can more efficiently improve the accuracy of machine learning models in areas lacking training examples by integrating artificial examples that satisfy similarity conditions. can do.
  • the generation unit 24 generates one artificial example for which one or more prediction results obtained using one or more machine learning models after training by the training unit 22 are uncertain, among the artificial examples.
  • an artificial example with an uncertain prediction result is an artificial example whose evaluation result of uncertainty satisfies a predetermined condition. The details of whether the uncertainty evaluation result satisfies the predetermined condition are as described above, and the details will not be repeated.
  • the generator 24 post-evaluates the uncertainties of the generated artificial examples using one or more trained machine learning models, and adopts artificial examples whose prediction results are uncertain through the post-evaluation.
  • FIG. 13 is a flow diagram illustrating the flow of the information processing method S20B according to the sixth exemplary embodiment.
  • the information processing method S20B shown in FIG. 13 is configured substantially in the same manner as the information processing method S20 according to exemplary embodiment 2, but differs in that step S205B is further included.
  • Step S205B In step S205B, the generation unit 24 post-evaluates the artificial examples generated in step S205.
  • the generation unit 24 evaluates the uncertainty of the prediction result for the artificial example using one or more machine learning models. For example, in the example shown in FIG. 6, the generation unit 24 evaluates the uncertainty of the prediction result for artificial example tv1-1 using machine learning models m1, m2, .
  • the details of the process of evaluating the uncertainty of prediction results using one or more machine learning models are as described in the second exemplary embodiment.
  • the generating unit 24 deletes the artificial example generated in step S205 when it is evaluated that the prediction result is not uncertain.
  • deleting an artificial example means deleting it from objects to be labeled in step S206 and objects to be added to training examples in step S208. As a result, artificial examples with uncertain prediction results are labeled and added to the training examples.
  • ⁇ Effects of this exemplary embodiment> a configuration is adopted in which the generation unit outputs artificial examples for which prediction results obtained using one or more machine learning models after training are uncertain, among the generated artificial examples. It is
  • artificial examples obtained by synthesizing training examples with uncertain prediction results do not necessarily have uncertain prediction results.
  • artificial examples generated in this way may not have uncertain prediction results.
  • Training a machine learning model using training examples whose prediction results are not uncertain is not efficient in improving the accuracy of the machine learning model.
  • the present exemplary embodiment generates artificial examples that can more efficiently improve the accuracy of machine learning models in areas lacking training examples by post-evaluating the generated artificial examples. be able to.
  • Some or all of the functions of the information processing apparatuses 10 and 20 may be realized by hardware such as integrated circuits (IC chips), or may be realized by software. good too.
  • the information processing device 10 and the like are implemented by a computer that executes instructions of programs, which are software that implements each function, for example.
  • An example of such a computer (hereinafter referred to as computer C) is shown in FIG.
  • Computer C comprises at least one processor C1 and at least one memory C2.
  • a program P for operating the computer C as the information processing apparatus 10 or the like is recorded in the memory C2.
  • the processor C1 reads the program P from the memory C2 and executes it, thereby realizing each function of the information processing apparatus 10 and the like.
  • processor C1 for example, CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit) , a microcontroller, or a combination thereof.
  • memory C2 for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.
  • the computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and temporarily storing various data.
  • Computer C may further include a communication interface for sending and receiving data to and from other devices.
  • Computer C may further include an input/output interface for connecting input/output devices such as a keyboard, mouse, display, and printer.
  • the program P can be recorded on a non-temporary tangible recording medium M that is readable by the computer C.
  • a recording medium M for example, a tape, disk, card, semiconductor memory, programmable logic circuit, or the like can be used.
  • the computer C can acquire the program P via such a recording medium M.
  • the program P can be transmitted via a transmission medium.
  • a transmission medium for example, a communication network or broadcast waves can be used.
  • Computer C can also obtain program P via such a transmission medium.
  • Appendix 1 an acquisition means for acquiring a plurality of training examples; A selection means for selecting two or more training examples for which one or a plurality of prediction results obtained using one or a plurality of machine learning models that output prediction results from the training examples as input are uncertain, from among the plurality of training examples; generating means for generating an artificial example by synthesizing two or more training examples selected by the selecting means; Information processing device with
  • the two or more training examples selected by the selection means include training examples in which there is variation in a plurality of prediction results obtained using a plurality of the machine learning models, The information processing device according to appendix 1 or 2.
  • the information processing device selects training examples in which a plurality of prediction results obtained using a plurality of machine learning models vary.
  • a machine learning model By training a machine learning model using artificial examples obtained by synthesizing training examples selected by an information processing device, it is possible to more effectively improve the prediction accuracy of the machine learning model to be trained.
  • the two or more training examples selected by the selection means include at least one training example near a decision boundary in the feature space of the machine learning model;
  • the selecting means selects, from among the plurality of training examples, training examples included in each of a plurality of spaces separated by decision boundaries in the feature space.
  • the information processing apparatus according to any one of Appendices 1 to 3.
  • the information processing device trains one or more machine learning models using the generated artificial examples, and generates artificial examples using one or more retrained machine learning models.
  • the prediction accuracy of the machine learning model to be trained can be improved more efficiently.
  • the generating means is generating a plurality of said artificial examples; 6.
  • the information processing apparatus according to any one of appendices 1 to 5, wherein two artificial examples satisfying a similarity condition among the plurality of artificial examples are integrated into one artificial example.
  • the generating means outputs one or more artificial examples for which one or more prediction results obtained using the one or more machine learning models trained by the training means are uncertain, among the artificial examples.
  • the information processing device according to appendix 2.
  • the information processing device performs post-evaluation of the artificial example using the machine learning model after training, and outputs the artificial example in which the actual prediction result of the machine learning model is uncertain.
  • the machine learning model to be trained can be trained more efficiently.
  • Appendix 8 wherein the one or more machine learning models comprises a training target machine learning model trained using the artificial examples; 8.
  • the information processing apparatus according to any one of Appendices 1 to 7.
  • the information processing apparatus 10 generates artificial examples by synthesizing one or more training examples with uncertain prediction results obtained using machine learning models that are training targets. This makes it possible to more efficiently improve the prediction accuracy of the machine learning model to be trained.
  • the selecting means selects, from among the plurality of training examples, two or more training examples for which a plurality of prediction results obtained using the plurality of machine learning models are uncertain, and 9.
  • the information processing apparatus according to any one of appendices 1 to 8, wherein at least two of the plurality of machine learning models use different machine learning algorithms.
  • the information processing device selects a training example to be synthesized using a plurality of machine learning models that use different machine learning algorithms.
  • various training examples are selected as training examples with uncertain prediction results, it is possible to prevent generated artificial examples from becoming uniform.
  • the selecting means selects, from among the plurality of training examples, two or more training examples for which a plurality of prediction results obtained using the plurality of machine learning models are uncertain, and 9.
  • the information processing apparatus according to any one of appendices 1 to 8, wherein each of the plurality of machine learning models uses the same machine learning algorithm.
  • the prediction accuracy of the machine learning model to be trained can be more efficiently improved.
  • Appendix 11 9. The information processing apparatus according to appendix 8, wherein at least one of the machine learning models to be trained is a decision tree.
  • the prediction accuracy of the decision tree can be improved more efficiently by training the decision tree using artificial examples generated by the information processing device.
  • Appendix 12 12. The information processing apparatus according to any one of Appendices 1 to 11, further comprising labeling means for labeling some or all of the plurality of training examples and the artificial examples.
  • a machine learning model can be trained using a training method that assumes that examples are labeled.
  • the generating means is a first generation process for synthesizing the plurality of training examples selected by the selection means; A second generating process of extracting at least one of the plurality of training examples selected by the selecting means, and synthesizing the extracted training example with a training example existing in the vicinity of the extracted training example in the feature space.
  • the first generation process and the second generation process are selected based on a predetermined condition, and the selected process is executed to repeatedly execute the generation process of generating an artificial example. This allows the information processing device to generate artificial examples having more diverse features.
  • the information processing device obtaining multiple training examples; Selecting two or more training examples in which one or more prediction results obtained using one or more machine learning models that output prediction results from the training examples as input are uncertain from the plurality of training examples, and selected two synthesizing one or more training examples to generate an artificial example; Information processing method including.
  • a program for causing a computer to function as an information processing device comprising: an acquisition means for acquiring a plurality of training examples; A selection means for selecting two or more training examples for which one or a plurality of prediction results obtained using one or a plurality of machine learning models that output prediction results from the training examples as input are uncertain, from among the plurality of training examples; generating means for generating an artificial example by synthesizing two or more training examples selected by the selecting means; A program that acts as a
  • Appendix 16 A computer-readable recording medium in which the program according to appendix 15 is recorded.
  • processor comprising: an acquisition process for acquiring a plurality of training examples; A selection process of selecting two or more training examples for which one or more prediction results obtained using one or more machine learning models that output prediction results from the training examples as input are uncertain, from among the plurality of training examples; and a generation process of synthesizing the selected two or more training examples to generate an artificial example.
  • the information processing apparatus may further include a memory, and the memory stores a program for causing the processor to execute the acquisition process, the selection process, and the generation process. good too. Also, this program may be recorded in a computer-readable non-temporary tangible recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)
PCT/JP2021/020115 2021-05-27 2021-05-27 情報処理装置、情報処理方法及びプログラム Ceased WO2022249373A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2021/020115 WO2022249373A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法及びプログラム
JP2023523844A JP7559943B2 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法及びプログラム
US18/561,357 US20240249205A1 (en) 2021-05-27 2021-05-27 Information processing apparatus, information processing method, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/020115 WO2022249373A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法及びプログラム

Publications (1)

Publication Number Publication Date
WO2022249373A1 true WO2022249373A1 (ja) 2022-12-01

Family

ID=84229561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/020115 Ceased WO2022249373A1 (ja) 2021-05-27 2021-05-27 情報処理装置、情報処理方法及びプログラム

Country Status (3)

Country Link
US (1) US20240249205A1 (https=)
JP (1) JP7559943B2 (https=)
WO (1) WO2022249373A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7694211B2 (ja) * 2021-07-06 2025-06-18 富士通株式会社 評価プログラム、評価方法および情報処理装置
US12443886B1 (en) * 2024-04-15 2025-10-14 NeuRealities, Inc. Techniques for adjusting mixed reality graphical environments using machine learning

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ANUBHA KABRA; AYUSH CHOPRA; NIKAASH PURI; PINKESH BADJATIYA; SUKRITI VERMA; PIYUSH GUPTA; BALAJI K: "MixBoost: Synthetic Oversampling with Boosted Mixup for Handling Extreme Imbalance", ARXIV.ORG, 3 September 2020 (2020-09-03), XP081754278 *
MACIEJ A. CZYZEWSKI: "batchboost: regularization for stabilizing training with resistance to underfitting & overfitting", ARXIV.ORG, 21 January 2020 (2020-01-21), XP081582844 *
WATANABE, MASATO ET AL.: "Unsupervised annotation target data selection method for active learning", IPSJ TECHNICAL REPORT, vol. 2018, no. 38, 19 September 2018 (2018-09-19), pages 1 - 8, ISSN: 2188-8701, Retrieved from the Internet <URL:https://ipsj.ixsq.nii.ac.jp/ej/?action=repository_uri&item_id=191383&fileid=1&file_no=1> *

Also Published As

Publication number Publication date
JP7559943B2 (ja) 2024-10-02
US20240249205A1 (en) 2024-07-25
JPWO2022249373A1 (https=) 2022-12-01

Similar Documents

Publication Publication Date Title
CN110462639B (zh) 信息处理设备、信息处理方法及计算机可读存储介质
US20250005360A1 (en) System and method for model parameter optimization
EP3605407A1 (en) Information processing device, information processing method, and computer-readable storage medium
CN114330736A (zh) 具有噪声对比先验的潜在变量生成性模型
US10373028B2 (en) Pattern recognition device, pattern recognition method, and computer program product
JP7485226B2 (ja) 訓練装置、分類装置、訓練方法、分類方法、及びプログラム
CN113487024B (zh) 交替序列生成模型训练方法、从文本中抽取图的方法
JP7559943B2 (ja) 情報処理装置、情報処理方法及びプログラム
CN115335831A (zh) 用于训练稀疏神经网络同时保持稀疏性的系统和方法
CN117242454A (zh) 通过查询嵌入在潜在嵌入集上的交叉注意力来生成神经网络输出
KR102689642B1 (ko) 확산 모델과 암시적 신경망 기반의 임의의 해상도와 스케일의 이미지를 생성하는 방법 및 이를 위한 장치
JP2024080175A (ja) 質問生成装置、質問生成方法、およびプログラム
JPWO2022249373A5 (https=)
JP2020027437A (ja) 学習装置および学習方法
JP7571879B2 (ja) 情報処理装置、情報処理方法、及びプログラム
JP6953376B2 (ja) ニューラルネットワーク、情報付加装置、学習方法、情報付加方法、およびプログラム
JPWO2022249392A5 (https=)
RU2718409C1 (ru) Система восстановления трехмерной структуры образца породы
JP7670156B2 (ja) 発明の名称 ： 情報処理装置、情報処理方法、及び、プログラム
JP7623986B2 (ja) 選別装置、選別方法、および選別プログラム
CN118628866A (zh) 基于稳定扩散模型的火星图像增广方法、计算设备和存储介质
CN114299517B (zh) 图像处理方法、装置、设备、存储介质及计算机程序产品
JP2022151502A (ja) プログラム、情報処理装置、及び方法
KR102321064B1 (ko) 부호화된 네트워크 생성 장치 및 방법
WO2023132029A1 (ja) 情報処理装置、情報処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943024

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023523844

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18561357

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21943024

Country of ref document: EP

Kind code of ref document: A1