WO2024096904A1 - Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl - Google Patents

Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl Download PDF

Info

Publication number
WO2024096904A1
WO2024096904A1 PCT/US2022/079099 US2022079099W WO2024096904A1 WO 2024096904 A1 WO2024096904 A1 WO 2024096904A1 US 2022079099 W US2022079099 W US 2022079099W WO 2024096904 A1 WO2024096904 A1 WO 2024096904A1
Authority
WO
WIPO (PCT)
Prior art keywords
pipeline
accuracy
pipelines
dataset
threshold
Prior art date
Application number
PCT/US2022/079099
Other languages
English (en)
Inventor
Yash GARG
Haoxiang Zhang
Original Assignee
Nokia Solutions And Networks Oy
Nokia Of America Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Solutions And Networks Oy, Nokia Of America Corporation filed Critical Nokia Solutions And Networks Oy
Priority to PCT/US2022/079099 priority Critical patent/WO2024096904A1/fr
Publication of WO2024096904A1 publication Critical patent/WO2024096904A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning

Definitions

  • the examples and non-limiting example embodiments relate generally to machine learning and, more particularly, to methods and apparatus for early termination of pipelines for faster training of AutoML systems.
  • BACKGROUND It is known to implement machine learning systems for classification and other data science tasks.
  • an apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: generate a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, select a pipeline from the set of pipelines, determine at least one accuracy from processing a dataset with at least one step of the selected pipeline, determine whether the at least one accuracy is greater than at least one threshold, terminate an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and select, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • a method includes generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, selecting a pipeline from the set of pipelines, determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, determining whether the at least one accuracy is greater than at least one threshold, terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • a non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable with the machine for performing operations, the operations including: generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, selecting a pipeline from the set of pipelines, determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, determining whether the at least one accuracy is greater than at least one threshold, terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • an apparatus includes: means for generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, means for selecting a pipeline from the set of pipelines, means for determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, means for determining whether the at least one accuracy is greater than at least one threshold, means for terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and means for selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • FIG.1A shows end-to-end evaluation of a machine learning model.
  • FIG. 1B shows early termination of a pipeline during evaluation of a machine learning model.
  • FIG. 2 shows a feedback loop between searching of pipelines and evaluation of pipelines.
  • FIG. 3 shows a detailed work flow of early termination of pipelines.
  • FIG. 4 depicts a method for implementing early termination of pipelines.
  • FIG. 5 is an example apparatus configured to implement the examples described herein.
  • FIG. 6 shows a representation of an example of non-volatile memory media.
  • FIG. 7 is an example method to implement the examples described herein.
  • the developers of the examples described herein have developed a set of algorithms and procedures for automated machine learning, aiming to simplify the time-consuming and complex steps of iterative model building to rapidly equip any engineering or business unit with powerful insights that can be obtained from data analytics activities.
  • the algorithms achieve this by automatically processing and featurizing the data for machine learning, and automatically applying relevant models to generate best predictions and insights without having the need for domain experts or data science experts.
  • the examples described herein are to be used to automate the model search for failure prediction on multi- variate time-series data received from loT sensors from a variety of users of enterprise solutions in diverse domains such as telecommunications, industrial, energy, transportation, etc.
  • AutoML automatic or automated machine learning
  • An important challenge is how to search the large space of ML pipelines in an efficient way.
  • An AutoML system typically works in the following iterative fashion: it first searches and generates a pipeline, which is then trained and tested in a second evaluation step. The result of the evaluation is then fed back into the first step to guide the search for another pipeline.
  • the major time cost comes from the evaluation of the end-to-end pipeline. Under a time budget, this evaluation overhead restricts the number of pipelines that can be explored, and thus results in a sub- optimal convergence, if not optimal, of AutoML systems.
  • eTOP early termination of pipelines
  • the eTOP strategy terminates a pipeline at a self-determined intermediate step within the pipeline when eTOP determines that the intermittent estimated accuracy will not improve.
  • eTOP can speed up the AutoML pipeline search process and accelerate the training of AutoML systems.
  • an automatic machine learning (AutoML) system requires multiple steps (computational operations), together referred as an AutoML pipeline, to solve a given problem. These steps can include data preprocessing, feature engineering, feature selection, and model training.
  • each of these steps has many choices to consider.
  • data encoding techniques for data-preprocessing and dimensionality reduction methods for feature engineering and many models to choose from, such as random forest, AdaBoost, and multi-layer perceptron, during the model training step.
  • AdaBoost AdaBoost
  • multi-layer perceptron during the model training step.
  • each method can come with its own hyperparameter choices making the search of a high performing AutoML pipeline computationally costly, both in time and resources, for example, the different number of layers and their respective sizes in an MLP-based model. This makes the search space very large, which makes it difficult for a data scientist or researcher (especially from non-technical domains) to manually find the best ML pipeline for a given problem.
  • AutoML systems aim to automate this search process and make data science more efficient, transparent and accessible.
  • this large search space poses challenges to AutoML systems as well, especially the major time cost in evaluating end-to-end pipelines. So given the same time budget, the more time an AutoML system spends on evaluating pipeline performance, the few pipeline search space it can explore, which can result in a sub- optimal convergence of the AutoML system on the given problem. [0019]
  • AutoML systems liberate data scientists from this tedious and time-consuming process.
  • the large and complex search space still exists, which makes the AutoML system computationally expensive.
  • a focus of AutoML systems is to optimize the pipeline searching time.
  • One way is to perform the optimization at two stages: at the offline stage, they identify a collection of proxy models for each machine learning algorithm; at the online stage, they evaluate these proxy models with a reduced dataset and select the model with the highest score for hyperparameter optimization.
  • the system may perform a two-stage optimization: first an offline stage, where the system acquires a portfolio of good-performing pipelines by metalearning experiments across a set of datasets; and second an online stage, where given a new dataset, all pipelines in the portfolio are evaluated one after the other and after these evaluations, the system continues with pipelines suggested by Bayesian optimization (BO).
  • the system may combine multiarmed bandits, Bayesian optimization and meta learning to optimize the search of pipelines.
  • the examples described herein target the acceleration of AutoML systems. It uses one or more surrogate prediction models to perform a step-wise condition check, so that mediocre pipelines are terminated at an earlier step without running the pipeline end-to-end, i.e. at steps.
  • eTOP starts from the given dataset and pre-defined pipeline search space to build a history of experiments for step- wise condition checkers. Then, for each pipeline, eTOP predicts the performance of this pipeline after each step and terminates the pipelines early that do not meet the performance criteria - as described herein.
  • FIG. 1A shows end-to-end evaluation 101 of a machine learning pipeline.
  • Input to the end-to-end evaluation 101 is a dataset 102, and the output is pipeline accuracy 112 for a classification task.
  • the end-to-end evaluation 101 includes data pre-processing 104, feature engineering 106, feature selection 108, and model training 110.
  • Data pre-processing 104 may include imputers and scalers.
  • Feature engineering 106 may include a OneHot encoder and dimension reduction.
  • Feature selection 108 may be based on a percentile or threshold.
  • Model training 110 may be based on Adaboost, a random forest, or a multi-layer perceptron (MLP).
  • MLP multi-layer perceptron
  • FIG. 1B shows early termination of a pipeline (103) during evaluation of a pipeline in an AutoML system.
  • step estimation based on data processed after each of data pre-processing 104, feature engineering 106, and feature selection 108.
  • accuracy A1 is estimated (105) after data pre- processing 104. If the accuracy A 1 is greater than a threshold, the method continues to feature engineering 106.
  • the accuracy A2 is estimated (107) after feature engineering 106. If the accuracy A 2 is greater than a threshold (which threshold may be the same or different than the one used for comparison with accuracy A1), the method continues to feature selection 108. Otherwise, the pipeline 100 terminates.
  • the accuracy A 3 is estimated (109) after feature selection 108. If the accuracy A3 is greater than a threshold (which threshold may be the same or different than the ones used for comparison with accuracy A1 and with accuracy A2) the method continues to feature selection 110. Otherwise, the pipeline 100 terminates.
  • the output (if not early terminated), is the pipeline accuracy 112.
  • FIG.2 shows a feedback loop between searching of pipelines 202 and evaluation of pipelines 206.
  • AutoML systems automate the search of pipelines 202, which search of pipelines 202 could include grid search or Bayesian optimization.
  • the feedback loop includes iteratively searching 202 (different AutoML systems use different search algorithms) based on the evaluation (208), and evaluating 206 based on the searching (204).
  • a subset of data samples D’ 304 is selected from an initial dataset D 302, and a subset of pipelines P’ 308 is selected from a set of pipelines P 306.
  • a history of experiments is generated from the subset of data samples D’ (304) and the subset of pipelines P’ 308.
  • the AutoML execution with early termination of pipelines (312) determines a winner pipeline 314, and based on the winner pipeline 314 a final model (316) is developed.
  • MetaLearning-Free the examples described herein do not require the time- consuming offline meta learning experiment to collect pipeline performance data. Instead, given a dataset D to study, the herein described system eTOP bootstraps with a certain amount of pipelines and the subset of data D to build a history by itself. Then with reference to this self-history, eTOP quickly goes over the rest of the pipelines in the search space to find the winner pipeline. [0031] 2. Intra-Pipeline: not like other search optimization methods that evaluate the pipeline from end-to-end, eTOP checks the stop condition for each step inside a running pipeline and terminates the pipeline immediately if the intermediate step does not meet the criteria. [0032] 3.
  • eTOP saves a great amount of time because of no need of the metalearning experiment. In addition, with the help of a hash map, eTOP can further speed up when going over the pipelines in the search space.
  • Fast Convergence instead of evaluating pipelines end-to-end, eTOP leverages surrogate models to estimate the pipeline performance after each step and terminates mediocre pipelines early. In this way, eTOP can help the AutoML system converge much faster.
  • the method 400 includes the steps presented in FIG. 4. Each of these steps is described in detail herein. [0035] Step 1 (401).
  • the input data to the methods is defined as dataset D and a set of pipelines P.
  • For the dataset D it is in the form of (features, target) as (X,Y).
  • Step 2 (402). Randomly select a subset of data D ' and a subset of pipelines P ' [0038] The goal of our method is to find good pipelines in a fast time. Given this purpose, a the subset dataset D ' is used, instead of the full dataset D, to evaluate each pipeline.
  • Step 3 (03).
  • Step 3 (03).
  • eTOP runs experiments as a collection of history H for later usage (i.e. for comparison in Step 9, 409).
  • the history ⁇ includes following information of ⁇ : ⁇ ⁇ , ⁇ ⁇ !, ⁇ ⁇ , ⁇ ⁇ , ⁇ ⁇ ! and ⁇ ⁇ , ⁇ ⁇ , ⁇ ⁇ , ⁇ ⁇ !, where A ⁇ is the accuracy evaluated after step ⁇ # .
  • Step 4 (404).
  • Step 5 (05).
  • ⁇ # get processed dataset + #
  • eTOP feeds it the data resulted from the previous step to obtain a new dataset + # processed at this step.
  • current step is OneHot Encoding and the previous step is Imputation of missing values.
  • eTOP uses the data resulted from Imputation to pass into OneHot Encoding, and then obtains a new set of data that have been processed by OneHot Encoding for use in a subsequent step such as a random forest algorithm.
  • Step 6 (406). If ⁇ ⁇ , ... , ⁇ # , ⁇ # ! in HashMap [0047] To decide whether to early terminate a pipeline ⁇ , eTOP needs to know if the estimated accuracy ⁇ # after step ⁇ # indicates that the current pipeline is a promising one (i.e. can achieve above average performance after the final step).
  • eTOP utilizes a hash map to store existing steps and accuracy information ⁇ ⁇ , ... , ⁇ # , ⁇ # !.
  • eTOP checks if the accuracy after these configuration of steps already exists in the HashMap or not.
  • HashMap (431) [0049] If the steps and accuracy information exists in the hash map, eTOP reads the estimated accuracy ⁇ # from the hash map (Step 7a, 407-a) and transitions directly to Step 9 (409). [0050] Step 8 (408). Predict the accuracy ⁇ # based on + # [0051] If the steps and accuracy information do not exist in the hash map, eTOP predicts the accuracy ⁇ # based on the data + # that are processed at ⁇ # . To perform the prediction action, eTOP utilizes surrogate models such as random forest, because it can provide a reliable prediction in a fast way. After the prediction, the steps and accuracy information ⁇ ⁇ , ... , ⁇ # , ⁇ # !
  • Step 11 (411).
  • Step 12 (12). Terminate the pipeline
  • the pipeline p is terminated if the accuracy is below the median accuracy from the history.
  • FIG. 5 is an example apparatus 500, which may be implemented in hardware, configured to implement the examples described herein.
  • the apparatus 500 comprises at least one processor 502 (e.g. an FPGA and/or CPU), at least one memory 504 including computer program code 505, wherein the at least one memory 504 and the computer program code 505 are configured to, with the at least one processor 502, cause the apparatus 500 to implement circuitry, a process, component, module, or function (collectively control 506) to implement the examples described herein, including early termination of pipelines for faster training of AutoML systems.
  • the memory 504 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g. ROM).
  • the apparatus 500 optionally includes a display and/or I/O interface 508 that may be used to display aspects or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc.
  • the apparatus 500 includes one or more communication interfaces (I/F(s)) 510 e.g. one or more network (N/W) interface(s).
  • the communication I/F(s) 510 may be wired and/or wireless and communicate over the Internet/other network(s) via any communication technique.
  • the communication I/F(s) 510 may comprise one or more transmitters and one or more receivers.
  • the communication I/F(s) 510 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder/decoder circuitries and one or more antennas.
  • the apparatus 500 may be part of a self-organizing/optimizing network (SON) node, such as in a cloud.
  • SON self-organizing/optimizing network
  • the apparatus 500 may also be distributed throughout a network including within and between apparatus 500 and any network element, such as a network control element (NCE) and/or a radio access network (RAN) node and/or a user equipment (UE).
  • NCE network control element
  • RAN radio access network
  • UE user equipment
  • Interface 512 enables data communication between the various items of apparatus 500, as shown in FIG. 5.
  • the interface 512 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like.
  • Computer program code 505, including control 506 may comprise object-oriented software configured to pass data and messages between objects within computer program code 505.
  • the apparatus 500 need not comprise each of the features mentioned, or may comprise other features as well.
  • FIG. 6 shows a schematic representation of non-volatile memory media 600a (e.g. computer disc (CD) or digital versatile disc (DVD)) and 600b (e.g.
  • FIG.7 is an example method 700 to implement the example embodiments described herein.
  • the method includes generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model.
  • the method includes selecting a pipeline from the set of pipelines.
  • the method includes determining at least one accuracy from processing a dataset with at least one step of the selected pipeline.
  • the method includes determining whether the at least one accuracy is greater than at least one threshold.
  • the method includes terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold.
  • the method includes selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • Method 700 may be performed with apparatus 500.
  • An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: generate a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, select a pipeline from the set of pipelines, determine at least one accuracy from processing a dataset with at least one step of the selected pipeline, determine whether the at least one accuracy is greater than at least one threshold, terminate an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and select, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • Example 2 The apparatus of example 1, wherein the apparatus comprises an automated machine learning system.
  • Example 3 The apparatus of any of examples 1 to 2, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine another accuracy from processing another dataset with another step of the selected pipeline, based on the at least one accuracy being greater than the at least one threshold.
  • Example 4. The apparatus of any of examples 1 to 2, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine another accuracy from processing another dataset with another step of the selected pipeline, based on the at least one accuracy being greater than the at least one threshold.
  • Example 5 The apparatus of any of examples 1 to 4, wherein the dataset comprises a result of processing of a previous step of the at least one step of the selected pipeline. [0078] Example 6.
  • Example 7 The apparatus of any of examples 1 to 6, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: generate a history of experiments using the dataset and the set of pipelines, the history of experiments comprising the at least one accuracy from processing the dataset with the at least one step of the selected pipeline.
  • Example 8 The apparatus of example 7, wherein the at least one threshold comprises a median accuracy from the history of experiments.
  • Example 10 The apparatus of any of examples 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine whether the at least one accuracy for a configuration of at least one step comprising the at least one step is in a hash map.
  • Example 11 The apparatus of example 10, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the at least accuracy from the hash map, based on the at least one accuracy being in the hash map.
  • Example 13 The apparatus of any of examples 1 to 12, wherein the dataset is constructed from data received from at least one internet of things sensor.
  • Example 14 The apparatus of any of examples 1 to 13, wherein the termination of the evaluation of the selected pipeline is performed without having performed an end-to-end evaluation of the selected pipeline. [0087] Example 15.
  • Example 16 The apparatus of any of examples 1 to 15, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: select a previously unselected pipeline for evaluation.
  • Example 17 The apparatus of any of examples 1 to 16, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: apply the machine learning model to at least one machine learning task.
  • Example 18 Example 18.
  • a method including: generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, selecting a pipeline from the set of pipelines, determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, determining whether the at least one accuracy is greater than at least one threshold, terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • a non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable with the machine for performing operations, the operations including: generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, selecting a pipeline from the set of pipelines, determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, determining whether the at least one accuracy is greater than at least one threshold, terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • Example 20 An apparatus including: means for generating a set of two or more pipelines, wherein a pipeline within the set of pipelines comprises a set of one or more steps used to learn a machine learning model, means for selecting a pipeline from the set of pipelines, means for determining at least one accuracy from processing a dataset with at least one step of the selected pipeline, means for determining whether the at least one accuracy is greater than at least one threshold, means for terminating an evaluation of the selected pipeline, based on the at least one accuracy being less than the at least one threshold, and means for selecting, from the set of pipelines, a winner pipeline to use for learning the machine learning model, the winner pipeline having an overall performance accuracy larger than another overall performance accuracy associated with at least one other pipeline of the set of pipelines.
  • references to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single/multi-processor architectures and sequential or parallel architectures but also specialized circuits such as field- programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other processing circuitry.
  • References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.
  • the memory(ies) as described herein may be implemented using any suitable data storage technology, such as semiconductor based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, non-transitory memory, transitory memory, fixed memory and removable memory.
  • the memory(ies) may comprise a database for storing data.
  • circuitry may refer to the following: (a) hardware circuit implementations, such as implementations in analog and/or digital circuitry, and (b) combinations of circuits and software (and/or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s)/software including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even if the software or firmware is not physically present.
  • circuitry would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and/or firmware.
  • circuitry would also cover, for example and if applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device.
  • arrows between individual blocks represent operational couplings there-between as well as the direction of data flows on those couplings.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Image Processing (AREA)

Abstract

Appareil comprenant des circuits configurés pour : générer un ensemble de deux pipelines ou plus, un pipeline compris dans l'ensemble de pipelines comprenant un ensemble d'une ou plusieurs étapes utilisées pour faire l'apprentissage d'un modèle d'apprentissage automatique ; sélectionner un pipeline dans l'ensemble de pipelines, déterminer au moins une précision par le traitement d'un ensemble de données avec au moins une étape du pipeline sélectionné, déterminer si la ou les précisions sont supérieures à au moins un seuil, terminer une évaluation du pipeline sélectionné, sur la base de la ou des précisions qui sont inférieures au ou aux seuils, et sélectionner, dans l'ensemble de pipelines, un pipeline gagnant à utiliser pour faire l'apprentissage du modèle d'apprentissage automatique, le pipeline gagnant ayant une précision de performance globale supérieure à une autre précision de performance globale associée à au moins un autre pipeline de l'ensemble de pipelines.
PCT/US2022/079099 2022-11-02 2022-11-02 Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl WO2024096904A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2022/079099 WO2024096904A1 (fr) 2022-11-02 2022-11-02 Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2022/079099 WO2024096904A1 (fr) 2022-11-02 2022-11-02 Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl

Publications (1)

Publication Number Publication Date
WO2024096904A1 true WO2024096904A1 (fr) 2024-05-10

Family

ID=90931234

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/079099 WO2024096904A1 (fr) 2022-11-02 2022-11-02 Procédés et appareil de fin anticipée de pipelines pour l'apprentissage plus rapide de systèmes automl

Country Status (1)

Country Link
WO (1) WO2024096904A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210081848A1 (en) * 2019-09-14 2021-03-18 Oracle International Corporation Techniques for adaptive pipelining composition for machine learning (ml)
US20210117841A1 (en) * 2020-12-23 2021-04-22 Intel Corporation Methods, apparatus, and articles of manufacture to improve automated machine learning
US20210224585A1 (en) * 2020-01-17 2021-07-22 NEC Laboratories Europe GmbH Meta-automated machine learning with improved multi-armed bandit algorithm for selecting and tuning a machine learning algorithm
US20220036246A1 (en) * 2020-07-29 2022-02-03 International Business Machines Corporation Automated machine learning pipeline generation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210081848A1 (en) * 2019-09-14 2021-03-18 Oracle International Corporation Techniques for adaptive pipelining composition for machine learning (ml)
US20210224585A1 (en) * 2020-01-17 2021-07-22 NEC Laboratories Europe GmbH Meta-automated machine learning with improved multi-armed bandit algorithm for selecting and tuning a machine learning algorithm
US20220036246A1 (en) * 2020-07-29 2022-02-03 International Business Machines Corporation Automated machine learning pipeline generation
US20210117841A1 (en) * 2020-12-23 2021-04-22 Intel Corporation Methods, apparatus, and articles of manufacture to improve automated machine learning

Similar Documents

Publication Publication Date Title
CN110807515B (zh) 模型生成方法和装置
CN111406267B (zh) 使用性能预测神经网络的神经架构搜索
CN110852421B (zh) 模型生成方法和装置
US11544536B2 (en) Hybrid neural architecture search
WO2018157752A1 (fr) Générateur de nombres aléatoires approximatif par fonction de distribution cumulative empirique
KR20200014510A (ko) 기계 학습 기반의 예측 서비스 제공 방법 및 그 장치
CN110852438A (zh) 模型生成方法和装置
CN111340221B (zh) 神经网络结构的采样方法和装置
WO2014186488A2 (fr) Adaptation des hyper-paramètres d'un algorithme d'apprentissage exécutable par ordinateur
CN111612134A (zh) 神经网络结构搜索方法、装置、电子设备及存储介质
US20210334700A1 (en) System and method of creating artificial intelligence model, machine learning model or quantum model generation framework
CN111340220B (zh) 用于训练预测模型的方法和装置
US20200167660A1 (en) Automated heuristic deep learning-based modelling
US20230108920A1 (en) System and method for providing robust artificial intelligence inference in edge computing devices
US20190228297A1 (en) Artificial Intelligence Modelling Engine
KR20220064866A (ko) 코스-파인 검색, 2-페이즈 블록 증류, 그리고 뉴럴 하드웨어 예측기를 이용한 하드웨어 및 뉴럴 네트워크 아키텍처들의 공동 설계를 위한 방법
CN111368973A (zh) 用于训练超网络的方法和装置
CN111353601A (zh) 用于预测模型结构的延时的方法和装置
CN114492601A (zh) 资源分类模型的训练方法、装置、电子设备及存储介质
CN111582488A (zh) 一种事件推演方法及装置
CN111373406A (zh) 使用问题匹配的先验知识提取的加速仿真设置过程
CN115905450A (zh) 一种基于无人机监控的水质异常溯源方法及系统
Sankar et al. Prediction of code fault using Naive Bayes and SVM classifiers
US20240095529A1 (en) Neural Network Optimization Method and Apparatus
CA3135954A1 (fr) Generation de code pour le deploiement d'un modele d'apprentissage automatique