JP2023004271A - Meta-automated machine learning with improved multi-armed bandit algorithm for selecting and tuning machine learning algorithm - Google Patents

Abstract

To provide a method for automatically selecting a machine learning algorithm and tuning hyper parameters of the machine learning algorithm.SOLUTION: The method involves receiving a dataset and a machine learning task from a user. Executions of multiple instantiations of different automated machine learning frameworks for a machine learning task are each controlled as a separate arm in terms of available computational resources and time budgets, and thereby multiple machine learning models are trained during execution by the separate arms and performance scores of the trained multiple models are calculated. One or more of the trained multiple models are selected for the machine learning task based on the performance scores.SELECTED DRAWING: Figure 1

Description

[0001] The present invention relates to machine learning (ML), and in particular to methods and systems for meta-automatic ML using multi-armed bandit algorithms for selecting and tuning ML algorithms.

background

[0002] Several high-level decisions must be made when applying ML. For example, the learning algorithm, or base learner, needs to be selected from many different available learning algorithms. Associated with each learning algorithm is another set of hyperparameters that can be optimized to maximize the algorithm's performance with respect to application-specific error criteria for a given dataset. Also, different feature preprocessing algorithms and feature selection techniques, each containing a set of hyperparameters, can be combined in the ML pipeline to improve the performance of the base learner. Different hyperparameters need to be tuned and different data preprocessing and feature engineering techniques may be applied accordingly. Automated machine learning (AutoML) explores automation of selecting base learners and preprocessors, and tuning associated hyperparameters.

[0003] First, AutoML is motivated by the goal of making ML accessible to non-experts. Second, AutoML is also motivated to make the process of applying ML more efficient, eg, to reduce the workload of skilled data scientists using automation. Third, AutoML hopes to provide a principled approach for applying base learners to ML problems (e.g., Mischa Schmidt et al., "On the Performance of Differential Evolution for Hyperparameter Tuning", arXiv; 1904.06960v1 (April 15, 2019)). Anh Truong et al., "Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tool," arXiv: 1908.05557v2 (September 3, 2019), incorporated herein by reference, iterates in the ML pipeline Describes the potential of AutoML to increase the productivity of data scientists, ML engineers and ML researchers using several different tools and platforms that seek to reduce tasks and thus automate repetitive tasks. there is

[0004] Several open source software frameworks for automating conventional ML, for example, Schmidt et al., "On the Performance of Differential Evolution for Hyperparameter Tuning," arXiv, incorporated herein by reference; 1904.06960v1 (April 15, 2019). Relevant scientific studies typically document feasibility and ML performance on a range of well-known test datasets, eg provided in the OpenML community. Frameworks such as these typically try to find the most appropriate ML algorithm for the user's ML task with best-performing hyperparameter settings (and train the selected and parameterized algorithm on the task's data). . Doing so is called algorithm selection and hyperparameter tuning. Additionally, the framework trains the selected and parameterized algorithms with the data of the ML task. Note that the above document does not describe how a non-expert user can easily invoke AutoML, but requires programming proficiency.

[0005] The subject of Neural Architecture Search (NAS) has been addressed in several publications, exclusively for deep learning. The frameworks described in these documents focus on devising neural network architectures by parameterizing deep learning libraries such as keras (keras.io) or tensorflow.

[0006] Recently, Micah J. et al., incorporated herein by reference. Smith et al., "The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development," arXiv: 1905.08942v3 (November 12, 2019) describes this framework, called AutoBazaar. , underlies the concept of ML pipeline templates by leveraging a variety of different existing ML and data manipulation libraries. This pipeline template is AutoBazaar's means of abstraction for algorithm selection and hyperparameter tuning. As such, AutoBazaar includes a step of selecting an algorithm (actually a pipeline selection among many possible candidate pipeline variants), a matched A technique (Algorithm 2) that iterates over successive steps of tuning the hyperparameters and training the pipeline (including the ML algorithm inside the pipeline) is described.

[0007] US Patent Application Publication No. 2016/0132787 describes the Cloud AutoML concept as follows. That is, the user defines data runs or tasks and enters them into the database. One of potentially many (in a cloud setting) worker nodes identifies, via a selection strategy, a so-called "hyperpartition" for which to tune hyperparameters. During tuning, the model has already been trained and a given set of Tested by dataset. The selection strategy may be uniform at random, or a standard multi-armed bandit algorithm (referred to as UCB1) to the performance score achieved so far, or a multi-armed multi-band that can handle floating rewards that indicate "BestK" and "BestKVelocity". One can either build one of two variants of the arm bandit algorithm. A hyperpartition is defined as the selection of hyperparameters for classification, eg which algorithm to run. To tune the hyperparameters, the commonly known Bayesian optimization via Gaussian processes is applied. By applying a selection strategy and then Bayesian optimization, the worker identifies which training job (ML algorithm/pipeline) should be applied next to the dataset and the corresponding training job specification into a central database. If a worker node is available (i.e. idle), the worker checks the central database, works on one of the potentially many training jobs in that database, and when it starts, the other workers do the same. Mark the training job to prevent working on it. Once ML is complete, the worker stores the performance and associated models and attempts to check the next training job or create a new training job for one of the specific data runs.

[0008] There are also AutoML as a Service (AMLaaS) offerings such as GOOGLE Cloud AutoML, which focuses on deep learning techniques, MICROSOFT AzureML, which leverages the AZURE ML algorithm, SALESFORCE TransmogrifAI, and UBER Ludwig. However, the inner workings of these mechanisms are not disclosed, so it is not known how these mechanisms scale their AMLaaS operations to the needs of millions of users. Nor is it known or disclosed how these mechanisms can leverage and match existing ML or deep learning algorithms.

overview

[0009] In one embodiment, the present invention provides a method for automatically selecting machine learning algorithms and tuning hyperparameters of machine learning algorithms. A dataset and a machine learning task are received from a user. The execution of multiple instantiations of different automated machine learning frameworks for a machine learning task are controlled as separate arms, each taking into account available computational resources and time budgets, thereby allowing separate A plurality of machine learning models are trained during execution by the arm, and a performance score of the trained models is calculated. One or more of a plurality of trained models for the machine learning task are selected based on the performance scores.

[0010] Embodiments of the invention are described in more detail below on the basis of exemplary figures. The invention is not limited to the exemplary embodiments. All features described and/or illustrated in this specification may be used alone or combined in different combinations in embodiments of the invention. Features and advantages of various embodiments of the present invention will become apparent upon reading the following detailed description with reference to the accompanying drawings illustrated below.

[0011] FIG. 1 illustrates the present invention, referred to herein as Hierarchical Automated Machine LEarning with Time-awareness (HAMLET) because of its ability to make hierarchical decisions and account for the passage of time. 1 schematically illustrates one embodiment of a system and method for meta-AutoML according to FIG.

[0012] Figure 2 is a diagram illustrating docker options.

[0013] Figure 3 is an exemplary graph illustrating how a learning curve (LC) is extrapolated based on observed learning curve values, according to one embodiment of the present invention.

[0014] Figures 4a, 4b and 4c show the first experiment as boxplots of HAMLET variant 1 rank with budgets of 15 minutes (Figure 4a), 30 minutes (Figure 4b), and 1 hour (Figure 4c). Moreover, it is a figure which each shows that the result does not change qualitatively with a small budget. See description of Figure 4a. See description of Figure 4a.

[0015] Figures 5a, 5b, 5c and 5d show the first experiment in HAMLET with budgets of 10 minutes (Figure 5a), 15 minutes (Figure 5b), 30 minutes (Figure 5c) and 1 hour (Figure 5d). Boxplots of variant 3 ranks and that the lower budget does not qualitatively change the results (10 min budget), respectively. See description of Figure 5a. See description of Figure 5a. See description of Figure 5a.

[0016] Figures 6a, 6b and 6c illustrate the first experiment inter-policy with budgets of 15 minutes (Figure 6a), 30 minutes (Figure 6b) and 1 hour (Figure 6c). Boxplots of the comparative ranks and the statistical indistinguishability of the policies at B=900 s, respectively. See description of Figure 6a. See description of Figure 6a.

[0017] Figures 7a, 7b and 7c show the second experiment as boxplots of HAMLET variant 1 rank at budgets of 2 hours (Figure 7a), 3 hours (Figure 7b), and 12 hours (Figure 7c). 4A and 4B are diagrams showing each; See description of Figure 7a. See description of Figure 7a.

[0018] Figures 8a, 8b, 8c and 8d illustrate the second experiment HAMLET with budgets of 1 hour (Figure 8a), 2 hours (Figure 8b), 3 hours (Figure 8c) and 12 hours (Figure 8d). FIG. 10 is a diagram showing each as a boxplot diagram of variant 3 ranks. See description of Figure 8a. See description of Figure 8a. See description of Figure 8a.

[0019] Figures 9a, 9b, 9c, 9d, 9e, 9f, 9g and 9h demonstrate that the second experiment was performed for 15 minutes (Figure 9a), 30 minutes (Figure 9b), and 45 minutes (Figure 9c) for 1 hour. (Fig. 9d), 2 hr (Fig. 9e), 3 hr (Fig. 9f), 6 hr (Fig. 9g), and 12 (Fig. 9h) budgets as boxplots of the ranking of inter-policy comparisons, respectively. is. See description of Figure 9a. See description of Figure 9a. See description of Figure 9a. See description of Figure 9a. See description of Figure 9a. See description of Figure 9a. See description of Figure 9a.

[0020] Figure 10 shows the 95% confidence intervals of the average rank of different policies, summarized for all datasets and budgets.

detailed description

[0021] Embodiments of the present invention present meta-AutoML systems and methods that leverage existing frameworks and libraries for algorithm selection and hyperparameter tuning, as well as for ensemble. These methods and systems use a modified multi-arm bandit algorithm augmented by learning curve extrapolation to predict the performance of each arm. The system is designed for convenient operability. Embodiments of the present invention are suitably designed to integrate the NAS framework conceptually as well as the AutoML framework of conventional ML algorithms.

[0022] Multi-armed bandit algorithms such as UCB1 can be used to learn to choose optimally among a large number of alternatives, but in general these algorithms assume a stationary reward distribution (essentially Ideally, the expected reward received by a multi-armed bandit algorithm for different arms should not change over time). However, in an AutoML setting, such as the hyper-partition of US2016/0132787 or the AutoML framework of embodiments of the present invention, the better the arm, the more computation time awarded (i.e., the more drawn), so the reward is non-stationary. US Patent Application Publication No. 2016/0132787 states that variants Best-K (reflects only a subset of the rewards received—the size of the subset is specified by the configuration parameter K) or Best-K-Velocity (rewards (reflecting differences in the best reward subset of K). However, these multi-armed bandit algorithms only look past and therefore do not adequately represent the problem.

[0023] With respect to providing AMLaaS and balancing workloads, known approaches are to provide AMLaaS using, for example, cloud computing, with the exception of the approach described in US Patent Application Publication No. 2016/0132787. , regardless of how it scales to provide US2016/0132787 uses a massively distributed architecture, which is compatible with cloud services. In this paper, workers pull their work from a central database, and the work is either running a single ML job (training a parameterized ML algorithm) or extracting data to determine hyper-partitions and hyper-parameterization. Means fetching a run and entering it into a database for other workers to fetch/work on. The approach of U.S. Patent Application Publication No. 2016/0132787 requires that jobs or data runs be assigned priorities and then workers select central database inputs, e.g., taking into account these priorities. do. The inventors have recognized that this is a disadvantage as the approach requires user interaction and knowledge regarding assigning priority. Furthermore, the approach of US2016/0132787 has certain properties/implications that are not favorable for AutoML problems. The following applies to all learning strategies of US2016/0132787 (i.e., non-uniform in random strategies that are themselves suboptimal since observations made during training are not available). That is, in this approach, one hyper-partition is chosen for data running using a selection strategy, specifically using a multi-armed bandit algorithm. Multi-armed bandit algorithms, such as UCB1, Best-K or Best-K Velocity, described in US Patent Application Publication No. 2016/0132787, use the observed experience (the performance of an already trained model of a possible different hyper-partition score). When the multi-armed bandit algorithm observes a new experience in the database (ie, a worker has finished a job and stored the model with its performance in the database), the multi-armed bandit algorithm updates its statistics. In the way US Patent Application Publication No. 2016/0132787 specifies the algorithm, the algorithm can observe new experiences only when the workers have finished training a model of the data run in question. After one hyperpartition is chosen, one tuning strategy is applied. Its tuning strategy is Bayesian optimization, which updates its probabilistic model with its associated hyper-partition experience already observed. These models are then used to identify the most likely hyperparameter configurations, eg, using well-defined expected improvement criteria. Bayesian optimization is only after choosing a hyperpartition, separate from the multi-armed bandit framework, and cannot be used to predict the performance of individual arms in advance. Traditionally, there is a single most likely hyperparameter configuration that workers then request execution via a central database. In summary, the mechanism of US2016/0132787 describes how multiple jobs for a single data run can be requested and executed (when the model needs to be updated). not

[0024] Embodiments of the present invention may utilize different frameworks. AutoBazaar, auto-sklearn, the framework of US Patent Application Publication No. 2016/0132787, etc. require tuning of the parameters of the ML algorithm and do not leverage different AutoML frameworks. In other words, known approaches only consider different ML algorithms and their hyperparameters, as opposed to considering different AutoML frameworks for algorithm selection and hyperparameter tuning. The inventors have discovered that this can lead to disadvantages for the following reasons. AutoML is still an active research field, with new ideas and open source frameworks being published frequently, and no clear state-of-the-art framework (Anh Truong et al., "Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools", arXiv: 1908.05557v2 (September 3, 2019)). Additionally, for different types of data sets and problems, different algorithms for hyperparameter tuning and algorithm selection may lead to the best results (e.g. Mischa Schmidt et al., "On the Performance of Differential Evolution for Hyperparameter Tuning", arXiv ; 1904.06960v1 (April 15, 2019)). Furthermore, running Gaussian processes, for example, to tune the hyperparameters of neural networks is still an active research area. As new algorithms are discovered or new frameworks are announced, the previous approaches described above cannot be easily incorporated into these systems. The previous approach had to solve the problem of that particular framework setting and implement that framework setting in the previous approach itself to suit the needs of that framework, e.g. There is a need to solve how Bayesian optimization methods can work efficiently for NAS. This means that new findings in research cannot be readily incorporated, leading to delays in improvement.

[0025] Additionally, it is not known how techniques such as those in US Patent Application Publication No. 2016/0132787 can incorporate AMLaaS frameworks such as GOOGLE AutoML. This can be a disadvantage, for example, because incorporating GOOGLE AutoML can lead to better results in terms of score as an open source framework.

[0026] Moreover, the previous approaches described above require programming skills to work with the AutoML framework, but there is no easy-to-use application programming interface (API). This reduces its usability for non-expert users, who build AutoML's key target audience.

[0027] Embodiments of the present invention overcome the problems and problems of previous approaches described above.

[0028] First, for the problem of non-stationary rewards, embodiments of the present invention present an extrapolated multi-armed bandit algorithm, which fits a learning curve function to the past rewards of an arm and its It means looking into the future by extrapolating the learning curve function to the end of the remaining time budget. Accordingly, embodiments of the present invention provide improved insight into how time budgets are allocated among alternatives as time progresses. A technique that examines the learning curve as it develops over runtime is recorded in a central database when compared to the "function call" based evaluation in US Patent Application Publication No. 2016/0132787 (and other work). Another aspect that is very useful is that the mechanism updates all statistics based on trained model performance, and these performance statistics are updated based on model performance after evaluation of the trained model. be. This implies that one model training (or multiple trainings if parallel training is performed) must finish before the statistics can be updated. The reason is that the performance statistics are based on the unit of "feature evaluation" (feature refers to training a parameterized algorithm on a dataset and recording its performance). This is because the multi-armed bandit algorithm and Bayesian optimization of US2016/0132787 operate on past samples (i.e., the multi-armed bandit algorithm and Bayesian optimization operate on new data (newly trained Only after looking at the model's performance) can we update its predictions and arrive at a better prediction). In embodiments of the present invention, the learning curve for the arms of a multi-armed bandit can be updated by exploiting learning curve extrapolation over time so that sampling occurs every x seconds and predictions can be updated. . Unless a new best model is reported with its score, the arm's performance simply remains unchanged as time progresses (and the budget is reduced). This means that embodiments of the present invention can stop execution of an arm at any time, and grant execution to another arm if the learning curve predicts that it will perform favorably. It also means that it can be assigned. In contrast, the approach of U.S. Patent Application Publication No. 2016/0132787 waits until the model finishes running to see changes in arm estimates, and then selects a multi-armed bandit algorithm (or Bayesian optimization). must be changed, which can take a long time.

[0029] Second, with respect to the challenges of providing AMLaaS and balancing workloads, embodiments of the invention are designed to scale from a single computer to a cloud setting, as described below. , thus overcoming the challenges of scaling to obtain operation in an AMLaaS setting. The difference between the approach of US2016/0132787 and the embodiments of the present invention in balancing the AutoML workload is obvious. That is, in the approach of U.S. Patent Application Publication No. 2016/0132787, workers pull work from a central database, and the work consists of running one machine learning job (training a parameterized machine learning algorithm), Or fetching data runs to determine hyperpartitions and hyperparameterization and input into a database for other workers to fetch/work on. In contrast, embodiments of the present invention use a dispatcher-master-worker concept, in which the master is assigned ML tasks by the dispatcher, the master assigns a time budget to the workers with whom the dispatcher works, Multiple workers can run in parallel as needed. This is useful for controlling resources per AutoML task (called "data run" in US2016/0132787). In the approach of US2016/0132787, jobs or data runs are assigned priorities, and workers are then required to select central database inputs, eg, taking these priorities into account.

[0030] Third, with respect to leveraging different frameworks, and in contrast to previous approaches discussed above, embodiments of the present invention use different AutoML frameworks (which internal algorithm selection and parameter tuning logic) can be leveraged, and this framework can be selected by an improved multi-armed bandit algorithm. Although conceptually simpler, the approach according to embodiments of the present invention is desirable because it overcomes the disadvantages discussed above. Additionally, embodiments of the present invention may incorporate AMLaaS frameworks such as GOOGLE AutoML, leading to better scores.

[0031] Fourth, with respect to ease of use for non-specialist users, embodiments of the present invention overcome the challenges discussed above by providing an easy-to-use API. Thus, embodiments of the present invention provide easier and better accessibility to ML.

[0032] In one embodiment, the present invention provides a method for automatically selecting a machine learning algorithm and tuning the hyperparameters of the machine learning algorithm. A dataset and a machine learning task are received from a user. Execution of multiple instantiations of different automated machine learning frameworks for a machine learning task are each controlled as a separate arm in view of available computational resources and time budgets, whereby during execution by separate arms Multiple machine learning models are trained and performance scores of the multiple trained models are calculated. One or more of the trained models for the machine learning task are selected based on the performance scores.

[0033] In one embodiment, the performance score is extrapolated to the rest of the time budget based on the achieved performance of each of the arms during the time interval of execution, which is part of the time budget.

[0034] In one embodiment, the method further includes allocating computational resources to the arm during the remaining time of the time budget based on the extrapolated performance score.

[0035] In one embodiment, the performance score is extrapolated by fitting a learning curve function to the past rewards of each of the arms and extrapolating the past rewards to the end of the remainder of the time budget.

[0036] In one embodiment, the method further comprises freezing execution of at least one of the arms based on the extrapolated performance score.

[0037] An embodiment further comprises resuming execution of at least one of the arms from the point at which the freeze occurred.

[0038] In one embodiment, at least some of the arms are performed by time division multiplexing using a selection mechanism to allocate computational resources to the arms during the time budget.

[0039] In one embodiment, at least some of the arms are executed in parallel.

[0040] In one embodiment, the method further comprises building an ensemble from the plurality of trained models.

[0041] In one embodiment, each of the arms is run as a microservice component of the cloud computer system architecture in a docker container with a respective container image of the automated machine learning framework.

[0042] In one embodiment, the docker container is housed within a larger docker container, which houses a separate docker container for the components that control the execution of the arm.

[0043] In one embodiment, the method comprises the steps of constructing a learning curve for each of the arms during the time interval of execution within the time budget, and extrapolating the performance score for each of the arms to the remainder of the time budget. and freezing or disabling execution of at least some of the arms based on the extrapolated performance score.

[0044] In one embodiment, a learning curve is constructed based on the maximum performance score achieved by each of the arms during the time interval.

[0045] In another embodiment, the present invention provides a microservices component encapsulated in a docker container of a cloud computing system architecture comprising one or more processors, the one or more processors independently or in combination, controlling the execution of multiple instantiations of different automated machine learning frameworks for a machine learning task, each as a separate arm given available computational resources and time budgets, thereby allowing separate arms to A plurality of machine learning models are trained during execution, and a performance score of the plurality of trained models is calculated to effect execution of the method including a controlling step.

[0046] In a further embodiment, the invention provides a tangible, non-transitory, computer-readable medium having instructions, the instructions being executed by one or more processors. results in, alone or in combination, execution of a method according to an embodiment of the present invention.

[0047] Embodiments of the present invention present a meta-AutoML system and method, referred to herein as HAMLET. HAMLET leverages the existing AutoML framework (the AutoML framework leverages existing ML libraries) to solve the user's task. HAMLET supports parallel execution of tasks for different users and supports disposition of different settings. HAMLET also comes with an easy-to-use API to support usage by non-expert users. Additionally, HAMLET can manage given time-budget constraints (and hardware constraints) by using a special multi-armed bandit algorithm.

[0048] HAMLET can be taken as an AutoML platform with the following properties to find the best feasible model for a given ML task. That is, HAMLET is
Automate algorithm selection and hyperparameter tuning of a very wide range of algorithms (by integrating different frameworks) for different types of ML tasks,
can be used by multiple (possibly non-expert) users with different hardware settings;
・・・Including time budget management,
- Making ML more accessible with an easy-to-use API.

[0049] Table 1 below shows the terms and definitions used to describe HAMLET herein.

[0050] FIG. 1 depicts the HAMLET system architecture. In one useful embodiment, the HAMLET User-Dispatcher, Dispatcher-HAMLET MasterBandit, and HAMLET MasterBandit-HAMLET arms of the interface are implemented as HTTP interfaces hosted in a standard web framework, such as Python's Flask. The interfaces to the depicted data storage components (database tables or separate databases) are based on standard techniques, for example Structured Query Language (SQL). It is possible to merge or separate data storage, or replace the depicted database with a file system based approach such as hadoop Distributed File System (HDFS). These implementation choices influence the technical implementation of the interfaces C1-C5, E1-E5 and F1-F3.

[0051] The dispatcher component is the point of contact for HAMLET users to request HAMLET services. Therefore, the dispatcher's interface A allows the user to
. . . upload datasets and define dataset descriptions;
Presenting a machine learning task description (e.g. performance function, budget, possibly which AutoML framework and configuration to use);
requesting the start of a configured task,
Receive information about the progress of specified machine learning tasks, e.g., performance scores and progress budgets;
optionally receive a notification when the training is over;
Receive an indication of which was the best performing model(s) or ensemble(s);
.optionally receives the trained model, e.g., a serial binary object, in a standardized format, e.g., a salted python object, and .in HAMLET, such as a unique identifier for the model in the HAMLET database. can be used to receive the model reference of the

[0052] The dispatcher registers a task with a MasterBandit and calls this MasterBandit to start training. To do so, the dispatcher passes the relevant configuration parameters (eg dataset reference task description, arm configuration) via interface B.

[0053] A HAMLET arm is an instantiation of an AutoML framework/algorithm for hyperparameter tuning and algorithm selection, which is, for example, an instantiation of a specific framework or of a framework with a specific user-defined configuration. can be made into There are usually multiple HAMLET arms. The HAMLET MasterBandit controls execution of HAMLET arms based on different decision rules (different embodiments). The HAMLET MasterBandit also manages the budget left for the task and checks whether the requirements (in terms of score to be reached) are met. Interface D conveys interaction between the MasterBandit and the arm, among other things:
. reporting performance scores and trained models;
Communicate starting, stopping, freezing, and continuing execution of the arm (and the AutoML process therein), and sharing configuration information.

[0054] The components are detailed below.
dispatcher component

[0055] A dispatcher receives a machine learning task that references a dataset, a description of the dataset, a loss function, stopping criteria, and the type of machine learning problem (regression, classification, or clustering) to which the task relates. The user also specifies to the dispatcher which variables are input and which of the data sets are output variables. Optionally, HAMLET can automatically infer what type of data format is contained in each column by applying programmatic heuristics from best practices apparent to those skilled in the art. can.

[0056] In one embodiment, the type of machine learning can be inferred directly from the dataset. A type of machine learning is considered a classification problem if all target variables labeled in the dataset description are categorical. On the other hand, if the target variable contains floating point numbers, the type of machine learning is a regression problem. Machine learning is a type of clustering when no target variable is included in the dataset.

[0057] Upon receiving a dataset, the HAMLET dispatcher stores the dataset in a dataset storage device (eg, a database table), assigns the dataset a unique dataset identifier that is returned to the user, and distributes the dataset to the user. associated with the identifier of Upon receiving the dataset description, the HAMLET dispatcher stores the dataset description in a description store (eg, database table), assigns a unique dataset description identifier to the dataset description, and associates the data description with the dataset identifier. Upon receiving the task specification, the HAMLET dispatcher stores the task specification in a task store (eg, database table) and assigns a unique task identifier to the task specification.

[0058] Different embodiments provide the following features.
In one particular embodiment, interface A for users to specify tasks (upload datasets) and for specifying dataset descriptions is a representational state transfer (REST) based interface. , conveys task descriptions and dataset descriptions in a standard format such as Extensible Markup Language (XML) or JavaScript Object Notation (JSON), and also allows dataset uploads through standard file upload mechanisms.
• In one particular related embodiment, the dataset is a tabular dataset stored in a comma separated value (csv) file format. Datasets are uploaded via interface A by standard file upload mechanisms known from Internet services.
... In one useful embodiment, HAMLET offers the user suggestions on which of the supported arms to consider in order to solve the task. Moreover, the user can also specify to HAMLET specific parameterizations of the components in the arm (eg, running algorithms, or specific hyperparameter ranges).
In one useful embodiment, when presented with a new user task, HAMLET can decide to disable arms that do not fit the task, or recommend arms that are particularly promising. HAMLET uses the ability to describe a task's dataset, eg, size, data types present in the dataset, whether there are missing features, and so on. Also, the type of machine learning task (eg, regression vs. clustering) is meaningful in considering decisions about which arm to apply. In order to identify meaningful arms to suggest, techniques known from the state of the art of "meta-learning" can be applied.
• In one useful embodiment, the user must provide authentication credentials before being able to use HAMLET.
Arrangement

[0059] In the following, we describe how the HAMLET design enables scaling from a single computer to a distributed cloud setting with huge numbers of parallel users and tasks.

[0060] Because task resolution can take a significant amount of time, and because many parallel tasks can be requested by different users, it is necessary to be able to scale system capacity. HAMLET is designed to be compatible with standard mechanisms for scaling up web services in cloud service settings, and is beneficially adaptable to support many concurrent user demands.

[0061] In one preferred embodiment of HAMLET, MasterBandit as a microservice component is encapsulated in, for example, a docker image. In this preferred embodiment, the cloud orchestrator component routes requests from the dispatcher to the MasterBandit. For example, Kubernetes can be used to manage the entire cloud system resources and among the resources many instances of the MasterBandit container. Similarly, one beneficial embodiment implements the HAMLET arm component as a microservice as a docker container. Each arm is a separate container. Thus, the HAMLET MasterBandit microservice can be implemented in one of two options.
1. If the microservice's own container environment provides a docker server instance, the docker server instance can run its arms inside the microservice's own container. In this option, share the virtualized cloud resources allocated to the MasterBandit container between the MasterBandit's main loop (see below) and different arm tuning calculations (see below), or2. Alternatively, if the cloud environment in which HAMLET is located provides docker services, MasterBandit can request start, freeze, stop and terminate arm containers as needed for its main loop. In this way, interface D's command set for controlling arm execution can be replaced with docker container execution control commands. Interface D is thus simplified.

[0062] FIG. 2 depicts, by way of example, two general options for placing the HAMLET. The letters 1 and N denote the normal cardinality of the relationship between the components. That is, there are multiple MasterBandit MBs associated with a single Dispatcher D, and multiple Arms A (depending on the task configuration) associated with the single MasterBandit MB. Requests between components can run through a docker orchestration component such as Kubernetes, or can use direct references to instantiated docker containers provided by docker service components. A single system can host a number of dispatchers instantiated for user requests by the Kubernetes orchestrator, for example for load balancing purposes, leveraging standard mechanisms defined in the prior art.
... In option 1, each of the microservice components is encapsulated in an individual docker container, but all docker containers are contained within one larger "external" docker container. In doing so, a single PC or small server may be a suitable deployment.
• In option 2, the dispatcher, MasterBandit and arm components are housed in individual docker containers.

[0063] In another embodiment, all microservice components (Dispatcher, MasterBandit, arms) are implemented in an operating system (OS) process, rather than relying, for example, on Docker as a virtualization technique, with associated standard mechanisms for execution runtime control. , or even a computing thread. In this embodiment, freezing arm execution may be accomplished via interface D as described above, or alternatively, standard operating system process control commands may be used to freeze and/or resume arm execution. used for

[0064] One particularly preferred embodiment relies on a combination of both encapsulations. That is, the dispatcher microservice component is encapsulated as a docker image. Another docker image bundles MasterBandit and the necessary framework libraries to implement the arm as a process inside the same docker container. In this way, arm execution control commands (start, stop, freeze and resume) in interface D are implemented by standard OS process control mechanisms which simplify interface D implementation. In a preferred arrangement, the dispatcher can then request the execution of MasterBandit in response to a user request via a docker orchestrator such as Kubernetes, for example.

[0065] In a preferred arrangement, the microservice component dispatcher can request the instantiation (and management) of a MasterBandit container from a docker server (on user demand) or an orchestrator such as Kubernetes.

[0066] In a preferred arrangement, the microservice component MasterBandit can request the instantiation (and management) of the appropriate number of arm containers from the docker server in order to solve the user's HAMLET task.

[0067] In one embodiment, different arm container images exist for different frameworks and are instantiated based on the HAMLET task specification. In this embodiment, MasterBandit must indicate the correct arm container type in its requests to the docker server. MasterBandit can also pass framework-specific configuration information related to HAMLET tasks (eg, hyperparameter ranges to use or algorithms to consider).

[0068] In another embodiment, a generic arm container is constructed with all possible frameworks supported by HAMLET and usable to start with dockerserver. In this variant, MasterBandit simply asks the docker server to start the desired number of arm containers and tells the containers the necessary Configuration is passed to the container along with framework-specific configuration information related to the HAMLET task for that arm (eg, hyperparameter ranges to use or algorithms to consider).
MasterBandit and arm component

[0069] Different existing frameworks for AutoML (e.g., auto-sklearn, or PMF, NAS (e.g., auto-keras), or dedicated machine learning tuning algorithms (e.g., Mischa Schmidt et al., "On the Performance of Differential Evolution for Hyperparameter Tuning”, arXiv; Integrated as arms that enter the HAMLET MasterBandit.The MasterBandit can choose from these selections or arms to solve specific user tasks.A different selection can be used for any given It may or may not be applicable to the user task.The HAMLET arm also integrates remote AutoML frameworks, such as Google Cloud AutoML, as tuners via the remote framework's client library (if available). Depending on the functionality of the remote framework, some of the execution embodiments below may not be feasible, for example, the GOOGLE AutoML framework does not support freezing execution of training.

[0070] For each task, a single MasterBandit component interacts with one or more arms, each from the AutoML and NAS mentioned above, and from the libraries of Cloud AML aaS services such as GOOGLE AutoML, its client libraries, or extracted by integrating specific customized versions of these external libraries. While running, the arm executes the library to user tasks. During execution, these libraries train a number of machine learning models and record associated scores. The performance scores of the trained models and the models themselves are stored in the database of FIG.

[0071] During arm execution, newly found models are continuously reported and stored as they are found, along with the score reached and the training time (the time required to find these models). The HAMLET MasterBandit can request scores and training times for each arm in each repetition.

[0072] The logic that controls the execution of solving the specified tasks resides in the HAMLET MasterBandit component of FIG. The MasterBandit decides on the resources to be used for the arms, ie which arms to run at which time intervals. MasterBandit can pause/resume arm running.

[0073] In one useful embodiment, MasterBandit runs arms in parallel for a configured or user-specified time budget. This setting is useful when huge computational resources are available to solve the user's task. With this setting, the MasterBandit does not have to choose between different arms for execution. In this embodiment, MasterBandit can monitor the execution (training of models from all arms) in the main loop or at specified time intervals until the relevant stopping criteria of the task are met. can. In one beneficial embodiment, MasterBandit can apply learning curve extrapolation (LCE), described below, for the purpose of presenting diagnostic information (eg, predictive performance of different arms) to the user.

[0074] In another embodiment, the MasterBandit component multiplexes different arms using computational resources of time through a suitable selection mechanism. For this, the MasterBandit determines the order in which the HAMLET arms can run. In this setting, it would be beneficial if MasterBandit applied machine learning to improve its decision performance. To do so, MasterBandit runs in a main loop for a period of time until the task's associated stopping criteria are met. In one variation, MasterBandit allows subsets of arms to be run in parallel.

[0075] In one particularly beneficial embodiment, denoted LCE, the HAMLET MasterBandit uses a novel and inventive improvement of the multi-armed bandit algorithm, in which all extrapolate the expected performance to the end of the task's remaining time budget for the arm of . This extrapolation is based on the arm's best performance achieved over the arm's individual execution time in the task. HAMLET chooses the one with the highest extrapolation performance. In this embodiment, only the best performance of different arms achieved over a period of time is considered in the extrapolation.

[0076] In one variant of the LCE embodiment (see Algorithm 1 below), when HAMLET decides to run one arm, HAMLET assigns a time interval (this interval can be a configuration parameter), Execution of the arm in that time interval, e.g. via an arm microservice (e.g. via interface D), a virtualized environment encapsulating the arm container, or via a configuration-related mechanism, e.g. obtained by process control Freeze the execution of MasterBandit checks the performance scores of the trained models, updates the learning algorithms (eg, multi-armed bandit statistics and associated extrapolation curves), and informs the selection step for the next loop iteration. If the same arm is chosen again in a later iteration, arm execution resumes and can start immediately where the previous arm was frozen, thus no computation time is lost. This approach is particularly useful. The reason is that while MasterBandit allows flexibility in assigning and reassigning arm runs, one arm has not yet increased its performance score, but as time progresses while running the arm, This is because the corresponding learning curve can be updated to consider whether computer resources should be reallocated.

[0077] In one particular embodiment, a multi-armed bandit algorithm, such as the well-known UCB1 algorithm (e.g., Micah J. Smith et al., "The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development", arXiv :1905.08942v3 (12 Nov 2019), provided by the BTB library, which selects which arm to run and learn based on the performance achieved by the selected arm. can be used to In another embodiment, the Best-K or Best-K-Velocity variant of the multi-armed bandit algorithm is used as the MasterBandit (e.g., Micah J. Smith et al., "The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System: Development, X 1905.08942v3 (provided by the BTB library referenced Nov. 12, 2019).

[0078] In another embodiment, if HAMLET decides to run one arm, HAMLET waits for the arm to complete training a preset number of models (eg, one). HAMLET checks the performance score of the trained model and updates the learning algorithm (eg UCB1 statistics or associated extrapolation curve for the multi-armed bandit algorithm) to inform the selection step for the next loop iteration. In this approach, an arm's learning curve is reported as the best performance for the number of models trained by different arms (in effect, e.g., reinterpreting model training as units of time instead of wall-clock time or CPU time). If so, use LCE.

[0079] In one embodiment, MasterBandit uses an arctangent function to fit and extrapolate the learning curve.

[0080] In one embodiment, MasterBandit uses a neural network to fit and extrapolate the learning curve. In one particular preferred embodiment, the neural network for learning curve extrapolation is pre-trained with exemplary learning curves from machine learning problems and datasets.

[0081] In one particular embodiment, MasterBandit considers only the convex hull of an arm's recorded performance scores, ie, the arm's best reported score, to construct the learning curve for each arm. In one beneficial embodiment, MasterBandit fills the time intervals between recorded arm scores by generating artificial performance score samples for the timestamps between recorded arm scores.

[0082] One particular approach to the MasterBandit's main loop is described in each of the following algorithms.
Algorithm 1:
Skip model persistence and statistics persistence for simplicity MasterBandit loops until user-specified task constraints are met.
This constraint can also be a time budget. On the first iteration, all arms are attempted once to give the MasterBandit the first set of performance scores.
. . After the first iteration, the MasterBandit:
. . . use the arm score to update the arm's learning curve . Extrapolate.
. . . The Master picks the most suitable arm from the learning curve and runs this arm for the configured time interval.
...a probabilistic search can be used by adjusting the ε parameter of Algorithm 1.2,
. . . Subtract the time interval from the remaining budget and check if the constraints are met.
• If not, repeat the loop, else stop training.
・While ConstraintMet=False:
Iteration k=1: each arm runs once for a given budget For each iteration k>1:
...For each arm a:
. . . Update LC_a(t)=[x_a=(t1,...,tnow), y_a=(score_t1,...,score_tnow)]
. . . Extrapolated_LC_a=UpdateExtrapolatedLearningCurve(LC_a(t), Budget_remaining)
... Next_arms, Next_Budget = MasterChooseArm(Extrapolated_LC forward each arm a, Budget_remaining, Desired_Score)
・・・Pause all arms not in Next_arms for Next_Budget seconds
・・・Resume all arms in Next_arms for Next_Budget seconds
. . . ConstraintMet=CheckConstraint(Budget_remaining, Desired_Score)
・・・ However, t: time,
...score: the score achieved with the best model ... LC: Learning_Curve
. , seconds the arm ran all the way until resp.skore was found for arm a)
y_a: score (monotonically increasing score found by arm a)
Next_arms: List of arms to run on next iteration if chosen by MasterBandit Next_Budget: Budget for next iteration, chosen by MasterBandit, seconds Algorithm 1.1
- Update Extrapolated Learning Curve (LC_a(t), Budget_remaining):
... the learning curve for arm a is extrapolated: Extrapolated_LC
... Extrapolated_LC estimates future scores over time up to the maximum available budget (Budget_remaining).
... the curve has been observed so far (y) over a period of time (t) by a standard regression algorithm, which means fitting the curve, for example using the ilog function or the arctangent function, or using the SVM for example. are fitted using a standard regression algorithm based on the scores obtained X=LC_a(t). x_a
y=LC_a(t). y_a
..Extrapolated_LC=curve_fit(X, y)
.Return Extrapolated_LC

Algorithm 1.2
- MasterChooseArm (Extrapolated_LC_a for each arm a, Budget_remaining, Desired_Score):
. . . Master chooses arm based on objective: Master can choose arm expected to obtain user-defined desired accuracy first (op=1), or with time remaining The arm expected to get the highest score can be chosen (op=2).
... The following shows the MasterBandit version, which, due to very limited resources, should only run one arm per repetition (the MasterBandit version runs multiple arms in parallel per repetition). can be adapted).
... the ε parameter controls the probabilistic exploration behavior.
. If op = 1
...with probability ε1:
...Next_arm=argmin_a (Extrapolated_LC_a.x where Extrapolated_LC_a.y>Desired_Score)
...with probability (1-ε1):
・・・Next_arm=random
. If op = 2
...with probability ε1:
. . . Next_arm=argmax_a (Extrapolated_LC_a.y where Extrapolated_LC_a.x=t_max_available)
...with probability (1-ε1):
・・・Next_arm=random
. . . Allocate the budget for the next iteration based on the remaining budget for each arm, the desired accuracy, and the extrapolated LC:
. . . Next_Budget=AssignBudgetArm(Budget_remaining, Desired_Score, Extrapolated_LC_a)
...Return Next_arm, Next_Budget

where x: time (relevant score training time, seconds the arm ran all the way until resp.skore was found for arm a)
y: score (monotonically increasing score found by arm a)

Algorithm 1.3
- CheckConstraint (Budget_remaining, Desired_Score):
...check if the full budget has been used and if the desired accuracy has already been achieved Constraint_met=False
. if Budget_used >= Budget_remaining:
. . . Constraint_met=True
if Highest_Score_Reached_above_all_arms>=Desired_Score:
. . . Constraint_met=True
.Return Contraint_met

[0083] In one beneficial embodiment, HAMLET converts a model's observed execution time for a portion of a particular data set (e.g., a test set commonly used to compute a performance score) into a model's performance score. In addition, track and store. This advantageously allows the user later to be presented with not only the best operating model, but also the fastest one to execute. It can also inform about ensemble construction, as indicated below.

[0084] When running against a user's task, HAMLET MasterBandit stores trained models and ensembles (see below) and achieves performance statistics in the corresponding database labeled in FIG. . Alternatively, the arm stores the trained model and associated performance statistics in the database itself during execution instead of leaving the storage in the MasterBandit.

[0085] Upon user request, the HAMLET dispatcher provides access to these stored models and statistics. In one useful variant, HAMLET may be configured to retain only a certain percentage or number of top behavioral models. This configuration can be provided by the user with the task definition, or it can be a HAMLET configuration parameter.

[0086] In one embodiment, MasterBandit launches a dispatcher to send the user, for example, an email address associated with the user or associated with the user's task (which was then provided as part of the task definition). , the user can be notified that the task has been completed.

[0087] In one embodiment, the MasterBandit is configured for allowable computing resource utilization by the human system administrator. In this case, the MasterBandit can select the appropriate mode (running all arms in parallel or multiplexing in time).

[0088] In another beneficial embodiment, the MasterBandit is informed of the allowable computing resource utilization by means of deployment orchestration technology (eg, the MasterBandit queries the Orchestrator, or the Orchestrator notifies the MasterBandit). , or by an appropriate cloud system load information service. In this case, the MasterBandit can switch between running all arms in parallel and multiplexing in time.

[0089] In one beneficial embodiment, HAMLET may be provided as a meta-AutoML cloud-based system.
HAMLET with ensemble

[0090] In one beneficial embodiment, HAMLET also provides for building ensembles based on all or a subset of the trained models for the task. Thus, models generated from different tuners (and frameworks) can be combined into an ensemble. Specifically, HAMLET can select a preset number or proportion of top performing models. In another variant, HAMLET can choose the fastest one to run the model. In yet another embodiment, HAMLET can select models based on the type of machine learning algorithm to diversify the algorithms that make up the ensemble. Three different embodiments are presented for scheduling the ensemble. i.e.
• In one embodiment, MasterBandit performs model ensemble computations after a task's stopping criteria are met.
... In another embodiment, during the main loop, MasterBandit performs the ensemble calculations at specified intervals, eg, every 5 iterations, or every 20 seconds.
• In a third embodiment, HAMLET during the main loop treats the model ensemble computation as a special form of tuner. That is, the model ensemble computation is treated as one of the MasterBandit's arms to be selected from the conclusions and implications discussed above.

[0091] In one particular embodiment, HAMLET uses the MetaBags concept for the ensemble computation (see Jihed Khiari et al., "MetaBags: Bagged Meta-Decision Trees for Regression," incorporated herein by reference). . For this reason, HAMLET generates different bootstrap samples of the task's dataset to compute MetaBags-specific meta-features. In another embodiment, HAMLET uses model averaging (averaging the predictions of all constituent models) as a means of ensemble.
Massive Concurrency, HAMLET with access to real-time budgets

[0092] In another embodiment, HAMLET is deployed in a cloud setting with abundant computing resources available to solve tasks for users with only a total wall clock budget specified. With this setting, the MasterBandit can run all its arms in parallel to be used in a given task, as described above for a given budget. However, the ARM framework can face unique challenges, for example, to scale well to many CPUs. Therefore, the HAMLET MasterBandit is
the configuration of each arm, e.g. by limiting the number of algorithms to be considered in the arm and/or by reducing the range of values of specific hyperparameters to be considered in the framework of the arm. , modifying each arm to focus on a limited search space, and . . . by adding complementary arms configured to compensate for the modified arm limitations, and/or may choose to compensate by adding the hyperparameter ranges removed by the modification from the modified arm configuration coverage.
HAMLET for specific use cases

[0093] A particular embodiment of HAMLET can be advantageously used for smart cities. A set of cameras that monitor pedestrian traffic feed into a demographic detection engine such as FieldAnalyst by NEC CORP. FieldAnalyst generates anonymous classifications of pedestrian demographic statistics (gender, age group) over time as they pass through a monitored area. This data, along with weather data and available data about upcoming events in the vicinity of the monitored area, serves as input to HAMLET for learning to predict future pedestrian traffic demographics. Upon solving the task, HAMLET produces a set of models that can predict future pedestrian demographics with some accuracy. Such predictions can then form the basis of dynamic traffic control decisions (e.g., to reduce traffic in the case of predictions), or stores can e.g. You can inform your marketing to prepare the crowd.

[0094] In another HAMLET embodiment for smart cities, a set of cameras monitoring road traffic can detect, for example, the number of cars, large vehicles (trucks, buses, etc.) and bicycles at certain time intervals. Populate the flow analytics engine. This traffic flow demographic data, along with weather data and other data about scheduled events in the city, predicts future traffic flow demographics, e.g., the mix of trucks, cars, and bicycles. serves as input to HAMLET for learning Upon solving the task, HAMLET produces a set of models that can predict future traffic flow demographics with some accuracy. Such predictions can then, for example, communicate with the navigation system provider to dispatch additional traffic police officers to manage traffic or, for example, to request rerouting of a certain percentage of drivers. can form the basis of dynamic traffic control decisions (eg, in the case of forecasting, to reduce traffic).

[0095] An embodiment of HAMLET can be advantageously used for predictive control of smart buildings for energy optimization. Smart buildings are equipped with sensors that measure room temperature, accessible via Internet of Things (IoT) infrastructure platforms, for example via FIWARE. Alternatively or additionally, the building's hydroponic heating system operating status (on, off, temperature) may be accessed, for example, from its building management system. This sensor data, together with relevant weather data, can be fed into HAMLET to identify highly accurate predictive machine learning models, e.g. hours or even days in advance (in particular weather forecast services from e.g. if used). In one useful variant, HAMLET can further identify machine learning models that can be applied to energy meter readings to predict how heating system operating settings and weather effects relate to heating system energy use. can. These predictive models (e.g., room temperature predictions and energy consumption) are then used by optimization algorithms such as genetic algorithms, differential evolution, or particle swarm optimization to determine under what heating system settings the building It can be evaluated whether a specific target room temperature range is met or violated and how heating system energy utilization can be optimized.

[0096] An embodiment of HAMLET can be advantageously used in hospitals to predict patient discharge. In hospital administration, it can be advantageous to know when a patient is likely to be discharged. This knowledge numerically encodes patient health data, physiological measurements (e.g. heart rate and blood pressure) and general patient information (e.g. age, gender) by methods known from the prior art, This can be accomplished by supplying HAMLET with this numerical data along with the number of days the patient has been hospitalized. HAMLET effectively creates a predictive machine learning model that can predict how long each patient will be hospitalized for new and already admitted patients. This discharge information can then be used, for example, for hospital resource planning.

[0097] An embodiment of HAMLET can be used to advantage in quantitative trading. Informing an investor's trading decisions feeds HAMLET with basic data on a security and a time series of its trading prices to create a) a model to classify which securities to buy or sell, and/or b) future securities. This can be accomplished by specifying a model that predicts the price (eg, the stock's closing price for the week from when the prediction was made).

[0098] An embodiment of HAMLET can be used advantageously in the electronic health domain. Given medical data, this data can serve as input to HAMLET for learning a disease classification based on that data. An example is as follows. That is, data collected at hospitals to analyze diabetes factors (eg, age, length of stay, medications, other illnesses, etc.). The task is to classify the risk of diabetic patient p having to be readmitted for an emergency diabetic case based on the given data. Solving this task, HAMLET produces a set of models that can classify a patient's urgent submission risk with a certain degree of accuracy. Such classifications can then be used to assist physicians in their decision-making and to provide hints and indications for specific treatments. This classification can also be applied to other disease cases.

[0099] An embodiment of HAMLET can be used advantageously in air quality prediction. A set of sensors that monitor road traffic and weather as well as air quality ₍ eg SO2 or particulate matter levels) at certain time intervals. This set of sensors serves as input to HAMLET for learning to predict future air quality, eg _SO2 or particulate matter levels. Upon solving the task, HAMLET produces a set of models that can predict future air quality values with some accuracy. Such predictions can then form the basis for dynamic traffic control decisions (eg, to reduce traffic in the case of predictions) or decisions about public transport. Another use of such predictions is to build an end-user application that informs the general public of the times when some area of interest is expected to be heavily contaminated.

[0100] In general and single use cases, embodiments of the present invention provide the following improvements/benefits.
1) The design of the meta-AutoML method, arming new AutoML frameworks (e.g. for NAS, or for conventional ML) as they are developed in the prior art, with efficient computational resources through time multiplexing. Easy integration, including novel and inventive improved multi-armed bandit algorithms to choose among different applicable AutoML frameworks (arms) by learning and extrapolating framework performance for use The design of this multi-armed bandit algorithm avoids the shortcomings of multi-armed bandit algorithms that are based on calculations about past performance statistics.
2) The design of the system, which is scalable from a single PC to a massively parallel cloud deployment with different hardware settings, and a framework (arm) running by using a flexible system architecture. or multiplexes the execution of the framework in time, and this system architecture allows, for example, bandits to manage the placement of arm microservice components (with budget knowledge). Designed to be encapsulated as
3) LEC provides an enhanced multi-armed bandit algorithm that improves the allocation of computing resources within the time budget of framework execution in scarce computing resource scenarios. Better scores can be found given limited resources, or desired scores can be found faster.
4) Deep learning as well as traditional ML support. It also supports different types of learning problems (regression, classification, clustering). Being able to use more algorithms leads to better scores.
5) Provides built-in ensemble of models across many different frameworks. This leads to better scores when models from across the framework are used (more good models incorporated lead to better scores).

[0101] In one embodiment, a method for automatically selecting, tuning, and training a machine learning algorithm for a user-specified machine learning task includes the following steps.
1) Receiving a dataset from a user.
2) Receiving a specific machine learning task from a user.
3) AutoML frames depending on available computational resources and time budget, with the goal of finding a trained model for a particular task, using performance learning and extrapolation against the remaining task time budget Controlling execution of multiple instantiations of works and/or algorithms (arms).
4) Collecting a number of trained models for a task for later user retrieval as well as to find the highest scoring model for a particular task.

[0102] Implementations of the present invention compared to user-invoked training of tasks by an open source framework integrated with HAMLET itself as an arm of HAMLET, and compared to not using a multi-armed bandit algorithm for resource management. Morphology offers several improvements. In the former case, the user may have to store and compare the resulting models themselves. This includes the following drawbacks.
... the decision as to which framework can be used for a given dataset has to be made by the user (losing the meta-AutoML effect of this step), and
• Parallel placement of frameworks requires more computational power than integration with the HAMLET multi-armed bandit algorithm, which focuses on probable framework placement (by extrapolation).
• Lower scores for the same amount of resources and/or more time and/or hardware resources for the same scores.
• The user can only choose one framework and invoke training with one framework (via the framework API) for a given problem. This may result in the best model (highest score) not being found or used.

[0103] Embodiments of HAMLET are described below, along with experimental results demonstrating the performance improvements and advantages provided by these embodiments. For example, automated algorithm selection and hyperparameter tuning facilitate the application of machine learning. HAMLET outperforms traditional multi-armed bandit strategies that look at the history of observed rewards to identify the most promising arms to optimize expected total rewards over time. The inventors recognized that these conventional strategies apply reward backward-looking when considering limited time-budgets and computational resources; Inappropriate because it doesn't look into the future to anticipate the highest final reward at the end. In contrast, HAMLET extends the bandit approach with learning curve extrapolation and computational time knowledge for choosing among a set of machine learning algorithms. In experiments with 99 recorded hyperparameter tuning traces from previous work, all HAMLET variants considered performed at least as well as other bandit-based algorithm selection strategies. The best-behavior HAMLET variant combines a learning curve search with the well-known confidence upper search bonus. In total, this variant achieves a lower average rank than all non-HAMLET policies with statistical significance at the 95% level.

[0104] HAMLET allows the selection of a base learner to be applied to a dataset. In one embodiment, an iterative approach is modeled to select a base learner and optimization of the hyperparameters of the base learner as a hierarchical problem. A multi-armed bandit focuses on selecting a base learner, and a dedicated component (also called a tuner) is responsible for tuning the hyperparameters of its respective base learner. This approach can be easily extended by using multiple base learners and merging them as additional arms. HAMLET is applied in practical settings where AutoML faces resource limitations such as available computing power and available time to solve machine learning problems. The following discussion deals with the extreme case of a single CPU that can be used to solve machine learning tasks within a tight wall clock budget. In this setting, the traditional multi-armed bandit approach is suboptimal. The reason is that this approach requires observing full functional evaluation, i.e. training a parameterization-based learner on the dataset to be able to update relevant arm statistics. It is from. In addition, most multi-armed bandit algorithms assume a fixed reward distribution, which is not true as tuning algorithms increase the achieved performance over time. Finally, in a typical AutoML setting, we do not want to maximize the average total reward over several iterations, but rather achieve the maximum possible performance. By explicitly accounting for time, HAMLET also learns the learning curve for the different arms and scales this learning curve at the end of the measured time budget with consideration of the computations already expended on each arm. Extrapolation leads to an improved multi-armed bandit approach. Combining learning curve extrapolation and accounting for computational time improves the performance of multi-armed bandits on algorithm selection problems.

[0105] The experimental evaluations discussed below use 99 sets of traces for six different base learners. The evaluation shows that even a simple technique of learning curve fitting yields improvements in areas of tight time budgets. Overall, the best performing HAMLET variant achieved better performance than all non-HAMLET bandits used in the experiment with 95% confidence.

[0106] The basic form of the multi-armed bandit problem can be described as follows. An agent repeatedly faces a selection among I different actions. After each choice, the agent receives a numerical reward chosen from a fixed probability distribution that depends on the action chosen. The objective is to maximize the expected total reward over a period or time step. Through repeated action selection, the agent can maximize gains by concentrating on the best arm. If an estimate of the action value is maintained, then any time step where the estimated value is maximum has at least one action, the greedy action(s). When one of the greedy actions is chosen, this choice is called exploitation. If a non-greedy action is chosen, this choice is called a search because it allows an improved estimate of the value of the non-greedy action. Search is required because there is always uncertainty about the accuracy of action value estimates. The greedy action currently looks best, but some of the other actions may actually be better.

[0107] A simple search technique would act greedily most of the time, but instead randomly select from all actions with small and equal probabilities, independent of the action value estimate. This method is called "ε-greedy". One advantage of ε-greedy is that the bandit samples every action an infinite number of times as the number of time steps increases. Therefore, the bandit's action value estimate will converge to the correct value. In general, the ε-greedy action selection forces non-greedy actions to be attempted, but indiscriminately. Another technique, denoted decaying ε, initializes ε high and slows ε (and thus the speed of the search) over time.

[0108] Another effective method of selecting among possible actions is the Upper Confidence Bound (UCB) method. This method selects actions by their potential to be optimal. UCB combines both its respective value estimates and the uncertainties in those estimates that actions have low value estimates or are selected less frequently over time than were already frequently selected. Take it into account and do so. The UCB bandit selects actions based on upper bounds on what is reasonable to assume as the highest possible truth value for each action. The perceptual uncertainty of the action value estimate should decrease each time an action is chosen. On the other hand, the cognitive uncertainty increases each time another action is chosen. One of the difficulties of UCB Bandit is dealing with non-stationary problems.

[0109] The multi-armed bandit problem presented and addressed by embodiments of the present invention differs from the original problem, where the reward is not fixed. When making algorithm selection, the reward should increase with more time spent in the arm, whereas the rate of improvement is unknown. The goal is to find the best single reward, not to increase the total reward.

[0110] Auto-Tuned Models (ATMs) are distributed collaborative, scalable AutoML systems that incorporate algorithm selection and hyperparameter tuning. ATM approaches AutoML by iterating in two steps: hyperpartition selection followed by hyperparameter tuning. A hyper-partition contains one specific base learner as well as its category of hyper-parameters. ATM models each hyperpartition selection as a multi-armed bandit problem. ATM supports three bandit algorithms: the standard UCB-based algorithm (called UCB1) and two variants designed to handle stray rewards encountered in the AutoML setting. Variants designed for float rewards have been used to select actions based on the velocity or mean of the best K-rewards observed so far (denoted as BestK-Velocity and BestK-Rewards, respectively). Calculate value estimates. Once a hyperpartition is chosen, the remaining unspecified parameters can be selected from the vector space using, for example, Bayesian optimization. The Machine Learning Bazaar framework for developing automated machine learning software systems extends ATM's work and incorporates the same bandit structure. HAMLET differs from both in many ways. First, HAMLET does not choose between hyperpartitions, it just chooses between base learners and thus does not choose hyperparameters of categories. Second, HAMLET uses a novel bandit algorithm that fits a simple model of the learning curve for the observed reward, but based on extrapolation of the learning curve to find the highest possible reward for a given time budget. to select an action. Third, ATM and Machine Learning Bazaar update action value statistics based on completed functional evaluations, ie test performance of base learners after training on datasets. In contrast, one embodiment of HAMLET updates training statistics at configurable time intervals. Even if the base learner's tuner is not managed to find a better model in recent time intervals, one embodiment of HAMLET tracks the tuner's learning curve progress (shortage) to avoid overshooting the learning curve. Allows switching resource allocation calculations based on

[0111] Hyperband is a bandit-based early stopping method for the sampled parameterization of the base learner. This method incorporates a bandit that addresses the fact that the arm in hyperparameter optimization can be improved given more training time. Hyperband is built on the concept of successive halves. That is, Hyperband runs a set of parameterization-based learners against a particular budget, evaluates their performance, and shuts down half of the worst performing set. When presented with a larger set of possible parameterizations of the base learner, Hyperband stops parameterizations that do not look promising and assigns successively more computational resources to the remaining ones. . HAMLET differs in that it allocates computational resources based on expected performance rather than historical past performance. In one embodiment of HAMLET, bandits are used to determine which algorithms to run, rather than which hyperparameter settings to run. Also, the method of allocating the budget is different. Hyperband applies the concept of geometric search to allocate an increasing portion of the total budget to a decreasing number of base learner parameterizations. In contrast, one embodiment of HAMLET continues the selected tuner for a set time interval. After this interval, the tuners report updates to the best model found, if any, and HAMLET updates the learning curve for each tuner.

[0112] The term "learning curve" refers to (1) the performance of an iterative machine learning algorithm as a function of the algorithm's training time or number of iterations, and (2) the performance of the machine learning algorithm as a function of data available for training. Used to describe as a function of set size. For the AutoML challenge addressed by one embodiment of HAMLET, its focus is on extrapolating the previous type of learning curve.

[0113] Targeting deep neural network (DNN) hyperparameter optimization is possible using a probabilistic model that extrapolates performance from the first part of the hyperparameter configuration learning curve. For this, a set of parametric functions can be fitted for each hyperparameter configuration and combined into a single model by weighted linear combinations. A stochastic extrapolation of the learning curve is obtained using the Markov Chain Monte Carlo method. These probabilistic extrapolations are then used for automatic early termination of runs with unpromising hyperparameter settings. It is also possible to extrapolate the hyperparameter settings and learning curve of the DNN's architecture. We can rely on Bayesian Neural Networks (BNN) in combination with known parametric functions to find samples as potential candidates for Hyperband application. Predicting the model parameters of the parametric function, as well as mixing the weights with the BNN, makes it possible to transfer knowledge about the previously observed learning curve. However, this implies that prior learning curve information is required to pre-train the BNN for good and fast performance.

[0114] A method known as Freeze-Thaw optimization is a Gaussian process-based Bayesian optimization technique for hyperparameter search. This method includes a learning curve model based on a parametric exponential decay model. A positive deterministic covariance kernel is used to model the iterative optimization curve. The Freeze-Thaw method maintains a set of partially completed but not actively trained models, and its learning curve model is determined at each iteration which should be "solved", i.e., should continue to be trained. used to make decisions.

[0115] Regression-based extrapolation models may be used for learning curve extrapolation to speed up hyperparameter optimization. This technique is based on using trajectories from previous builds to make predictions for new builds, where "build" refers to a training run with a specific base learner parameterization. Therefore, it is possible to transform data from previous builds, add a noise term to match the current build, and extrapolate its performance. This extrapolation function can serve to identify and stop hyperparameter configurations early.

[0116] In one embodiment of HAMLET, a less sophisticated learning curve function is used to estimate the general nature of the benefits obtained by moving from a backward-looking multi-armed bandit to a forward-looking multi-armed bandit for algorithm selection. Clarify. Also, one embodiment of HAMLET presents a general approach that is not limited to a particular type of base learner such as a DNN. No previous learning curve was relied upon in demonstrating the effectiveness of an embodiment of HAMLET. Transferring information from previous learning curves is clearly beneficial, but this suggests that HAMLET improves algorithm selection performance not because of transferring and reusing previous information, but because of simple learning curve extrapolation. specify that In one embodiment, HAMLET uses information only from current AutoML problems. Also, one embodiment of HAMLET extrapolates the learning curve of the performance of the tuner of the base learner rather than the individual hyperparameter configurations.

[0117] Table 2 below presents a list of symbols and notations used below to further describe possible implementations of HAMLET.

[0118] According to embodiments of HAMLET, the AutoML algorithm selection problem is modeled as a multi-armed bandit problem, with each arm representing a hyperparameter tuner responsible for one specific base learner. At each iteration, HAMLET chooses which action to take based on an extrapolation of the arm's learning curve, described below and outlined in Algorithms 2 and 3, ie chooses which arm to draw. After determining the action, the bandit continues execution of the corresponding hyperparameter tuner for a preset time interval Δt. After this interval, arm execution is paused, but can be resumed at that point in a later iteration without loss of information. During execution of the arm within that time interval, the bandit collects all monotonically increasing accuracy values reached within that time interval, as well as information about when these accuracies were reached (i.e., updated so far). learning curve). If the tuner does not find a new monotonically increasing accuracy value within that time interval, this information is also taken. Bandit then fits a parametric curve to match the learning curve and reduces the remaining budget by Δt. HAMLET then proceeds to the next iteration. When HAMLET faces a new AutoML problem, it preferably tries arms in a round-robin fashion until HAMLET collects enough values to model a learning curve for each arm.

[0119] Learning curve extrapolation: According to HAMLET embodiments, the learning curve is modeled as the accuracy found by the tuner (y) over the training time (x) in seconds. where training time is the time spent running the tuner to specify the parameterization of the base learner and the time spent training the parameterized base learner on the dataset including. In HAMLET, the learning curve is defined as a monotonically increasing function defined by the maximum accuracy found over training time. An exemplary graph for demonstrating the learning curve extrapolation is shown in FIG. In this example, the arm exercise is running for 500 seconds and the maximum exercise time for this arm is marked with a vertical line. Accuracy values found up to this point are used to fit a curve in order to predict future accuracy values over time. From the accuracy scores found by this tuner, only monotonically increasing values are used for the learning curve, other scores are ignored. For comparison, an actual future learning curve is also shown, which is the ground truth of the extrapolated learning curve, but unknown to the HAMLET bandit. One embodiment of HAMLET finds the base learner that optimally uses a specified time budget B and achieves the highest accuracy by dedicating the most computational resources to the base learner's corresponding tuner. make it a goal Therefore, HAMLET tries to predict the maximum achievable accuracy for each tuner. HAMLET does so by extrapolating each tuner's learning curve assuming that all remaining budget _{B_rem} has been spent on each tuner. Considering that each tuner i has already received some training period time t _x ⁱ , HAMLET uses the tuner's learning curve to determine x=t _x Predict accuracy at ⁱ +B _rem .

[0120] To investigate whether learning curve extrapolation is a meaningful concept for the algorithm selection problem in a constrained computational setting, a direct parametric function was used to observe the tuner over time. modeled accuracy. In the problem setting according to embodiments of HAMLET, the learning curve is known to be (1) monotonically increasing, (2) saturating the function, and (3) with values y ε '0,1'. Because of the similar form and possible fulfillment of preconditions (1)-(3), according to one embodiment, we use an arctangent function with four parameters (a, b, c, d) , it is chosen to transform, decompress and compress this first set of studies. i.e.
y=a arctan(b(x+c))+d (1)
Here, a scikit-learn curve fitting function is used to fit the parameters of the desired curve.

[0121] HAMLET variant: HAMLET faces the same search/exploitation dilemma as other multi-armed bandit strategies. Below we describe three variants of how HAMLET chooses which arm to run on the next iteration. In each, the determination is based on the value of Berthol r, which includes the predicted accuracy for each arm i by extrapolating the learning curve to time x=t _x ⁱ +B _rem ,r ⁱ . Three variants are outlined in Algorithm 3.

[0122] HAMLET Variant ₁ Double ε _- greedy learning curve extrapolation with fixed ε1 and ε2: In this approach, HAMLET operates in an ε-greedy method based on learning curve extrapolation. After observing in preliminary experiments that a subset of tuners often performed much better than the rest, the standard ε-greedy bandit was modified as follows. That is, at chance ε2, HAMLET randomly chooses _one action. At chance ε ₁ , HAMLET chooses the tuner with the second highest prediction accuracy. At chance 1-(ε ₁ +ε ₂ ), HAMLET takes a greedy action, ie argmax(r).

の繰り返しステップでε（Ｂ）＝０まで低減させ、ここで表記ε（ｔ）は、確率的探索パラメータの時間依存性を表示する。各繰り返しにおいて、ＨＡＭＬＥＴは、チャンスが現在ε（ｔ）で１つのアクションをランダムに選ぶ。チャンス１－ε（ｔ）では、ＨＡＭＬＥＴは貪欲アクションを取る。 [0123] HAMLET Variant 2 ε-greedy learning curve extrapolation with decay: In this approach, HAMLET operates in an ε-greedy method based on learning curve extrapolation. A variant starts with ε(t)=1 and sets this ε to

in repeated steps until ε(B)=0, where the notation ε(t) denotes the time dependence of the probabilistic search parameters. At each iteration, HAMLET randomly chooses one action whose chance is now ε(t). At chance 1-ε(t), HAMLET takes greedy action.

ここで、ｎは合計繰り返し数、ｎ^ｉは腕ｉが引き出された回数、ρは探索ボーナスの倍率である。各繰り返しにおいて、ＨＡＭＬＥＴは、最大

を持つ腕を選択する。

[0124] HAMLET VARIANT 3 Learning Curve Extrapolation with Exploration Bonus: This variant adds one scaled UCB-based exploration bonus to the learning curve prediction per arm and calculates the score as follows: .

where n is the total number of iterations, ni is the number of times arm ⁱ has been drawn, and ρ is the exploratory bonus multiplier. At each iteration, HAMLET

Select the arm with the

[0125] Tracing an experiment in which hyperparameter tuning was performed on six base learners by an evolutionary strategy (Mischa Schmidt et al., "On the Performance of Differential Evolution for Hyperparameter Tuning", arXiv; 1904.06960v1, (April 2019) 15)) is used and is referenced in the discussion below. Running different algorithm selection policies on recorded experimental traces allows different bandit policies to be evaluated based on ground truth. These traces are from a classification dataset. Equation (1) above can be adjusted for other data sets, such as regression data sets.

[0126] Computational resources and setup: Each tuner (and base learner) ran in a single docker container, accessible only to a single CPU. Parallel execution of different experiments was limited to ensure that a full CPU core was available to each docker container. There was no limit on memory resource availability. In one embodiment of HAMLET evaluated below, the bandit logic also runs in a docker container constrained to a single CPU core. The run was a limited and different experimental run to ensure that each docker container had access to one full CPU core.

[0127] Datasets, base learners and hyperparameter tuners: Referring to the traces from experiments used and discussed below, the algorithm selection is 49 small (OpenML datasets: {23, 30, 36, 48 , 285, 679, 683, 722, 732, 741, 752, 770, 773, 795, 799, 812, 821, 859, 862, 873, 894, 906, 908, 911, 912, 913, 932, 943, 971 , 976, 995, 1020, 1038, 1071, 1100, 1115, 1126, 1151, 1154, 1164, 1412, 1452, 1471, 1488, 1500, 1535, 1600, 4135, 40475}) and 10 large (OpenML data Set: {46, 184, 293, 389, 554, 772, 917, 1049, 1120, 1128} iterations (5 each) were performed on the data set. Table 3 below, with the validation experiment parameter set, records the budgets used for the algorithm selection experiments for the small data set (labeled Experiment 1) and the large data set (labeled Experiment 2). The recorded traces contain six base learners: k-nearest neighbors, linear and kernel support vector machines (SVM), Adaboost, random forest, and multi-layer perceptron.

[0128] Policy and parameterization for comparative evaluation: For HAMLET variants 1-3, the time is Δt = 10 seconds, with the ability to freeze and continue execution of different arms (e.g., via standard process control mechanisms). interval. For fair comparison with other bandit policies, the experiment takes into account the computational time required to fit the arm's learning curve. In this work, HAMLET variants 1-3, a simple round-robin strategy (“Round Robin”), standard UCB1 Bandit (“UVB”), BestK-Rewards (“BestKReward-K”, where K is used parameter selection) and BestK-Velocity (“BestKVelocity-K”) policies to leverage the BTB library. HAMLET variant 1 is presented in "MasterLC-ε ₁ -ε ₂ ". HAMLET variant 2 is referenced by “MasterLCDecay” and “MasterLC-UCB-ρ” refers to HAMLET variant 3. For each parametrizable policy (BestK-Rewards, BestK-Velocity, HAMLET variants 1 and 2), a simple grid search was run to identify the best operating parameters for that policy given the entire dataset and the entire budget. (See Table 3). This simple grid search mimics a realistic setting where data scientists may not know the optimal policy parameterization in advance and therefore parameterize based on educated guesses.

[0129] Analysis: The highest accuracies achieved by each policy parameterization for each dataset within _a given budget _were compared using boxplot plots to determine the of the most promising selects were identified. After identifying the best operating policy parameterizations for each policy across different budgets, these best operating policy parameterizations were compared to each other. The 95% confidence intervals for the average rank of policies in these inter-policy comparisons were then calculated. In the figures discussed below, IQR stands for interquartile range. Interquartile ranges represent the middle 50% of the data. The interquartile range is related to the quartile as it is the difference between the third quartile and the first quartile of the data. In a ranked dataset, quartiles are three values that divide the dataset into four equal parts. Each of the four parts contains 25% of the data. Quartile 1 is the smallest quartile. 25% of the dataset is below quartile 1, 75% of the dataset is above quartile 1, and so on. In a ranked dataset, quartiles are three values that divide the dataset into four equal parts. Each of the four parts contains 25% of the data. Quartile 1 is the smallest quartile. 25% of the dataset is below quartile 1, 75% of the dataset is above quartile 1, and so on. The ranking is sorted so that the best result gets the lowest rank, the second best result gets the second lowest rank, and so on. The worst results get the highest rank. Experimental results are discussed below.

[0130] Figures 4a, 4b and 4c show boxplots of the HAMLET variant 1 ranks of Experiment 1, showing that a significant level of constant randomness and too small a level of probabilistic search are detrimental to variant 1 performance. It is confirmed that ε ₁ =0.1 and ε ₂ =0.1 were chosen among similarly performing parameterizations for comparison with other policies. With a smaller budget, the results do not change qualitatively.

[0131] Figures 5a, 5b, 5c and 5d show boxplots of the HAMLET variant 3 ranks of Experiment 1, for scaling the UCB exploration bonus, medium to large rho inactivates the UCB bonus at all. , it is confirmed that the performance of variant 3 is not favorable. ρ=0.05 was chosen for comparison with other policies. With a smaller budget, the results do not change qualitatively.

[0132] Figures 6a, 6b and 6c show selected boxplots of small data set ranks for different budgets for inter-policy comparison. In particular, it can be seen that HAMLET variants 1 and 3 achieve good performance across all experiments.

[0133] Similar to Figures 4a, 4b and 4c, Figures 7a, 7b and 7c confirm that high levels of constant randomness and too small levels of probabilistic search are detrimental to Variant 1 performance. be. A selection of ε ₁ =0.1 and ε ₂ =0.1 was confirmed among the parameterizations.

[0134] Similar to Figures 5a, 5b, 5c and 5d, Figures 8a, 8b, 8c and 8d show that medium to large p for scaling the UCB search bonus is not favorable for Variant 3 performance. is confirmed. ρ=0.05 was confirmed.

[0135] Figures 9a, 9b, 9c, 9d, 9e, 9f, 9g and 9h show box plots of the ranks of the larger dataset for all eight budgets. HAMLET variant 1 (MasterLC-ε ₁ -ε ₂ ) and variant 3 (MasterLC-UCB-I) achieve good performance. With a higher budget, BestKRewards-7 and UCB can catch up.

[0136] HAMLET variants 1 and 3 benefit from time awareness and learning curve extrapolation capabilities. Extrapolation is beneficially encouraged but discouraged because too high levels of chance or search bonuses reduce algorithm selection performance (Figs. 4a-c, 5a-d, 7a-c and 8a-c). d). In each experiment, HAMLET variants 1 and 3 performed better than the competitive policy at some low to medium budgets. For budgets outside that range, HAMLET variants 1 and 3 perform at least as well as the BestK-Reward policy.

[0137] The boxplots of Figures 6a-c and Figures 9a-h demonstrate that the HAMLET variant performs better than the compared policies for a range of budgets. Figure 10 shows the aggregation of all 1485 runs (99 traces x 15 budget levels). Here, HAMLET variant 3 learning curve extrapolation combined with search uncertainty bonus is 95% better than other policies (except HAMLET variant 1), especially since their respective confidence intervals (CI) do not overlap. Achieving statistically significant and good performance at level. Therefore, our findings that the combination of learning curve extrapolation and accounting for computational time improve the performance of multi-armed bandits in algorithm selection problems are certainly valid, from experiments and in particular HAMLET It can be concluded from the performance of variant 3. Given that the applied learning curve extrapolation technique is straightforward (see equation (1) above), the more Sophisticated learning curve techniques can also be applied.

[0138] It was also verified that the trends in the results did not change when the best performing policies of the different policy groups were compared to each other by budget. Finally, it was observed during experiments that the BestK-Velocity policy usually performed much worse than the BestK-Rewards or UCB strategies.

[0139] In summary, a range of bandit policy parameterization experiments show that even a straightforward approach to extrapolating the learning curve can be a useful refinement to the bandit-based algorithm selection problem. Statistical analysis indicates that the HAMLET variant is at least similar to the standard Bandit approach while offering several improvements and advantages detailed herein. Notably, HAMLET variant 3, which combines learning curve extrapolation and a rescaled UCB search bonus, is shown in FIG. As shown, it outperforms all non-HAMLET variants. Even further performance improvements can be made, e.g., by using more sophisticated learning curve modeling techniques, e.g., using BNN-based learning curve predictors, or by integrating meta-learning concepts, e.g. It can be achieved by letting

[0140] While embodiments of the present invention have been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. is. It should be understood that changes and modifications can be made by those skilled in the art within the scope of the appended claims. In particular, the invention encompasses further embodiments together with any combination of features of the different embodiments described above and below. Additionally, statements made herein that characterize the invention refer to one, but not necessarily all, embodiments of the invention.

[0001] The terms used in the claims should be understood to provide the broadest reasonable interpretation consistent with the foregoing description. For example, use of the articles [a] or [the] when introducing an element should not be construed as excluding more than one element. Similarly, the recitation of "or" means that the recitation of "A or B" is the same as that of "A and B," unless it is clear from the context or the preceding description that only one of A and B is intended. should be construed as inclusive so as not to exclude Further, references to "at least one of A, B and C" are to be interpreted as one or more of the group of elements consisting of A, B and C, where A, B and C are It should not be construed as requiring at least one of each of the listed elements A, B and C, whether categorical or otherwise related. Further, references to "A, B and/or C" or "at least one of A, B or C" refer to any singular entity, e.g. It should be construed to include any subset, e.g. A and B, or the entire list of A, B and C from the elements listed.

Claims (15)

A method for automatically selecting a machine learning algorithm and tuning hyperparameters of said machine learning algorithm, comprising:
receiving a dataset and a machine learning task from a user;
controlling the execution of multiple instantiations of different automatic machine learning frameworks for said machine learning task, each as a separate arm given available computational resources and time budgets, thereby during execution by said separate arms; a controlling step in which a plurality of machine learning models are trained in and a performance score of the plurality of trained models is calculated;
selecting one or more of the plurality of trained models for the machine learning task based on the performance score;
A method, including 2. The method of claim 1, wherein the performance score is extrapolated to the remainder of the time budget based on the achieved performance of each of the arms during a time interval of execution that is part of the time budget. 3. The method of claim 2, further comprising allocating the computational resource to the arm during the remaining time of the time budget based on the extrapolated performance score. 3. The performance score is extrapolated by fitting a learning curve function to the past rewards of each of the arms and extrapolating the past rewards to the end of the remainder of the time budget. The method described in . 3. The method of claim 2, further comprising freezing the execution of at least one of the arms based on the extrapolated performance score. 6. The method of claim 5, further comprising resuming said execution of at least one of said arms from the point at which said freezing occurred. 2. The method of claim 1, wherein at least some of the arms are performed by time division multiplexing using a selection mechanism to allocate the computational resources to the arms during the time budget. 2. The method of claim 1, wherein at least some of said arms are executed in parallel. 2. The method of claim 1, further comprising building an ensemble from the plurality of trained models. 2. The method of claim 1, wherein each of said arms is implemented as a microservices component of a cloud computer system architecture in a docker container with a respective container image of said automated machine learning framework. 11. The method of claim 10, wherein the docker container is housed within a larger docker container, the larger docker container housing a separate docker container for a component that controls execution of the arm. constructing a learning curve for each of said arms during a time interval of said execution within said time budget;
extrapolating the performance score of each of the arms to the remainder of the time budget;
freezing or disabling execution of at least some of the arms based on the extrapolated performance score;
2. The method of claim 1, further comprising: 13. The method of claim 12, wherein the learning curve is constructed based on maximum performance scores achieved by each of the arms during the time interval. A microservice component encapsulated in a docker container of a cloud computing system architecture comprising one or more processors, said one or more processors, alone or in combination, comprising:
controlling the execution of multiple instantiations of different automated machine learning frameworks for a machine learning task, each as a separate arm given available computational resources and time budgets, thereby during execution by said separate arms A microservice component configured to effect execution of a method comprising a controlling step in which a plurality of machine learning models are trained and a performance score of said plurality of trained models is calculated. A tangible, non-transitory computer-readable medium having instructions that, when executed by one or more processors, alone or in combination:
controlling the execution of multiple instantiations of different automated machine learning frameworks for a machine learning task, each as a separate arm given available computational resources and time budgets, thereby during execution by said separate arms A tangible, non-transitory computer-readable medium for effecting execution of a method comprising a controlling step in which a plurality of machine learning models are trained and performance scores of the trained models are calculated.

Images