CN110048886A - A kind of efficient cloud configuration selection algorithm of big data analysis task - Google Patents

Abstract

The invention proposes a kind of efficient clouds of big data analysis task to configure selection algorithm, small-scale cluster experiment is carried out by selected part input data, and then construct performance prediction model, utility prediction model estimates performance of the task on large-scale cluster, and passage capacity prediction result determines optimal cloud configuration.By using above-mentioned algorithm, can be configured with the lower model training time and at effectively helping user to find optimal cloud originally.Large-scale data analysis task to be deployed on cloud computing platform selects optimal cloud configuration, can significantly improve its operational efficiency, and reduce operating cost.

Description

An efficient cloud configuration selection algorithm for big data analysis tasks

technical field

The invention belongs to the field of cloud computing, and in particular relates to an efficient cloud configuration algorithm based on big data analysis tasks.

Background technique

Large-scale data analysis tasks are growing and involve increasingly complex tasks, often involving machine learning, natural language processing, and image processing. Compared with traditional computing tasks, such tasks are usually data-intensive and computationally-intensive, requiring longer computation time and higher computational cost. Therefore, in order to complete large-scale data analysis tasks, the huge computing power of cloud computing is usually used to help complete the task. Choosing the best cloud configuration for large-scale analysis tasks can improve the operation efficiency of the task and reduce the computing cost of users.

To meet different computing requirements, existing cloud service providers provide users with hundreds of instance types with different resource configurations (such as Amazon's EC2, Microsoft's Azure, and Google's ComputeEngine). While most cloud service providers only allow users to select instance types from a pool of available instance types, Google's Compute Engine allows users to custom configure virtual machines (configuring vCPUs and memory), which also makes choosing the right cloud configuration difficult more challenging. In addition to this, major cloud service providers also offer serverless cloud architectures (such as Amazon Lambda, Google Cloud Functions, and Microsoft Azure Functions), which allow users to run tasks as serverless functions without using pre-specified configurations Start the instance. However, serverless architectures may require applications to refactor their code, and serverless cloud providers are not able to help users minimize task completion time or help users reduce computing costs.

The choice of cloud configuration, that is, the choice of instance type and number of instances, directly affects the completion time and economic cost of tasks. A properly chosen cloud configuration can achieve the same performance goals at a lower cost. Uncovering potential cost savings is even more important due to the longer runtime of large-scale data analysis tasks. Due to the diversification of tasks and the combination of instance types and cluster sizes, the search space for cloud configurations becomes huge.

In such a large search space, the use of exhaustive searches for optimal cloud configurations is neither practical nor scalable. To limit the search space, the CherryPick algorithm selects the optimal cloud configuration by limiting the search space with limited task information. CherryPick is optimized for cost minimization, but cannot be used to optimize other goals, such as minimizing job completion time through cost budgeting. In addition to this, Ernest and PARIS use a performance modeling approach to select cloud configurations. By using such performance prediction models, users can choose different cloud configurations for tasks with different optimization goals, such as choosing the cheapest or fastest cloud configuration. However, Ernest needs to train a predictive model for each instance type, while PARIS only selects the best instance type across multiple public clouds without giving the cluster size.

SUMMARY OF THE INVENTION

Aiming at the deficiencies of the prior art, the present invention proposes an efficient cloud configuration selection algorithm for big data analysis tasks.

The technical solution of the present invention is an efficient cloud configuration selection algorithm for big data analysis tasks, comprising the following steps:

Step 1: The training data collection stage, the implementation method is as follows,

The training data collector conducts experiments with specific instance types on only a small portion of the input data, which will be used to predict the performance of the task execution on the entire input data. Training data collection includes experiment selection and experiment execution.

Experiment selection: In the experiment selection, two important experimental parameters need to be determined: (1) the ratio, that is, the ratio of the experimental data to the total input data; (2) the number of cloud server instances used for task execution. The present invention uses statistical techniques to select part of the experimental parameters, and mainly uses the experimental parameters that can generate as much information as possible to predict the performance of the task running, thereby ensuring high prediction accuracy. According to D-optimality, choose the experimental parameters that maximize the weighted sum of the covariance matrix (information matrix). The experimental parameter settings are denoted using E _i =( _xi , _yi ), where _xi is the number of instances and _yi is the input data scale. Let M denote the total number of experimental parameter settings obtained by enumerating all possible scales and instance numbers. Then, using E _i , a K-dimensional feature vector F _i can be calculated, where each term corresponds to a term in the prediction model. In this way, M eigenvectors are obtained for all experimental settings. According to D-optimality, when selecting experimental parameters, choose to maximize the covariance matrix (information matrix) The experimental parameters of the weighted sum, i.e. The constraints are 0≤α _i ≤1,i∈[1,M], where α _i represents the probability of choosing i experimental setting. The total cost of the experiment is represented by adding a budget constraint term B, where y _i / _xi is the cost of running the experiment E _i according to the pricing model on the cloud platform. When solving the above optimization problem, the M experimental settings are sorted in non-increasing order according to the probability α _i and the top data set is selected as the training data. In the present invention, the first 10 data groups are selected as training data.

Experiment Execution: After a selected experiment setup, determine which data samples from the entire input dataset are used to make up the experimental dataset to satisfy the specified scale. In the present invention, random sampling is used to select data samples from the entire input data set, because random sampling can avoid falling into an isolated area of the data set. After obtaining a small dataset, deploy the specified number of instances using the chosen experiment settings and start running the task, after which the experimental parameters and task completion time are used as training data for building the predictive model.

Step 2: Model construction stage, the implementation is as follows,

Model Builder consists of Model Builder and Model Converter. Using the training data collected for a specific instance type, the model builder can build a base predictive model. After that, the model transformer transforms and derives the prediction models of the remaining instance types according to the base prediction model.

Model Builder: When running experiments on a subset of the input dataset on a specific instance type, use T _base (x,y) to represent the task runtime, given the number of instances x and the scale of the dataset y. Large-scale analysis tasks are usually run in successive steps (i.e. iterations) until a termination condition is met. Each step mainly consists of two phases: concurrent computation and data communication. The computational time of task execution remains relative to the dataset size, and there are several representative communication patterns in large-scale analysis tasks. Therefore, the running time of large-scale analysis tasks is inferred by parsing the computation time and communication time. The main objective of the present invention is to obtain the performance prediction function T _base (x, y) for a given task by calculating and communicating modes of tasks and designing fitting terms involving x and y.

Computational time-consuming, the time cost incurred by the user-defined iterative algorithm operating on each sample of the input data. For large-scale data processing tasks in a cluster computing environment, the computation time can be approximated by several different fitting terms, depending on the characteristics of the dataset (eg, dense or sparse) and the algorithm. Thus, the computation time will be a function of the number of instances and the size of the dataset.

Communication is time-consuming, and the time cost of data transmission to the target node through the network. Figure 1 abstracts representative communication patterns in large-scale data analysis tasks. Although there are differences in programming models and execution mechanisms, common communication patterns represent most communication situations in clustered applications. The communication time is mainly a function of the number of instances, and the fitting term of the function can be inferred according to the different communication modes of the task. For example, when the data size of each instance is constant, the communication time increases linearly with the number of instances in the partition-aggregate communication mode, but it is a quadratic relationship for the shuffle communication mode.

Given all candidate fit terms of the function T _base (x,y), mutual information is used as the selection criterion for the fit terms, redundant terms are excluded and only good predictors are selected as the fit terms. Assume represents the set of all candidates, where each term fk is a function of _x and y determined by the computation and communication mode. Given m training data samples collected at different numbers of instances and different data scales, first compute the K-dimensional feature vector F _i = (f _1,i ,...,f _K,i ) for each experimental setting, e.g. f _k,i =y _i / _xi . Then, the mutual information between each item and the running time is calculated, and the items whose mutual information with the running time is higher than a threshold are selected. According to m training runtime samples, the basic prediction model is obtained by fitting The value of w _k in . Wherein β _k indicates whether the fitting item f _k is selected (β _k =1 indicates that this item is selected).

Model converters: Cloud providers typically offer various instance families with different combinations of CPU, memory, hard disk, and network capacity to meet the needs of different jobs, such as general purpose and compute/memory/storage optimized. Extensive experiments show that given a task and a fixed dataset, the runtime of one instance type can be converted to a different instance type based on a simple mapping. Therefore, there is no need to perform experiments on each instance type to obtain training data before building a predictive model, which greatly reduces training time and training cost.

The transformer Φ is the mapping from the base prediction model to the target prediction model Φ:T _base (x,y)→T _target (x,y). By comparing the running times of different instance types under the same task and dataset size, the categories of fit terms in the prediction function are similar. In other words, under the same task and dataset size, if f _k is contained in T _base (x, y), it is likely that T _target (x, y) should also contain f _k . This is mainly because the computation and communication patterns of tasks remain largely unchanged under the same application configuration and number of instances. However, under different instance types, the weight of each item will be different, so we need to pay attention to the weight mapping from the base prediction model to the target prediction model. A simple and effective mapping method is adopted in the present invention. Assume represents the lowest cost of the experimental settings chosen by the training data collector, with a running time of t _base . We run experiments on the target instance type to get the running time t _target . The model transformer exports the prediction model of the target instance type as in

Step 3: Selector construction stage, the implementation is as follows,

Integrate runtime prediction models for all instance types into a single runtime predictor T(x,y), where x is a cloud configuration vector consisting of the type and number of instances. For a given input dataset of tasks, the goal is to enable the user to find the most preferred cloud configuration that satisfies certain runtime and cost constraints. Let P(x) be the price per unit time of cloud configuration x, that is, the unit price of the instance type multiplied by the number of instances. The optimal cloud configuration selection problem can be formulated as x ^* =S(T(x,y),C(x),Ry, where Cx=Px×Tx,y,0≤y≤1

where C(x) is the price per unit time of cloud configuration x and R(y) is a user-added constraint such as the maximum tolerable runtime or the maximum tolerable cost. The selector S(*) is determined by the user and is used to select the best cloud configuration x ^* that satisfies the desired performance or cost.

Description of drawings

FIG. 1 is a schematic diagram of the communication mode of the present invention.

Fig. 2 is the overall design structure diagram of the present invention

Fig. 3 is the effectiveness comparison diagram of the present invention

Fig. 4 is the prediction accuracy rate on Spark of the present invention

Fig. 5 is the prediction accuracy rate on Hadoop of the present invention

Fig. 6 is a comparison diagram of the total task time of the present invention and the model training time

Fig. 7 is the prediction accuracy rate of TeraSort of the present invention under different data set sizes

Figure 8 is the cost of WordCount of the present invention on different instance types

Figure 9 is the completion time of TeraSort and WordCount of the present invention on different cluster sizes

Detailed ways

The present invention mainly proposes an efficient cloud configuration selection framework for big data analysis tasks based on the computing mode and communication mode of big data analysis tasks, so that users can find cloud configurations suitable for a given big data analysis task, thereby greatly reducing large-scale data The computational cost of the analysis task. This framework builds predictive models through a small number of experiments, using very little input data and small-scale clusters, and can convert a predictive model of one instance type to a predictive model of another instance type with very little additional experimental intelligence. The invented cloud configuration selection framework allows cloud computing users to determine the best cloud configuration at a lower cost.

Referring to Fig. 2, the embodiment takes the cloud configuration selection algorithm (named as Silhouette) of the big data analysis task realized on the Amazon cloud service (Amazon Web Service, AWS) as an example to specifically illustrate the process of the present invention, as follows:

Step 1: The training data collection stage, the implementation method is as follows,

The training data collector conducts experiments with specific instance types on only a small portion of the input data, which will be used to predict the performance of the task execution on the entire input data. Training data collection includes experiment selection and experiment execution.

Experiment selection: In the experiment selection, two important experimental parameters need to be determined: (1) the ratio, that is, the ratio of the experimental data to the total input data; (2) the number of cloud server instances used for task execution. In this embodiment, a statistical technique is used to select some experimental parameters, and the experimental parameters that can generate as much information as possible are mainly used to predict the performance of the task when running, so as to ensure high prediction accuracy. According to D-optimality, choose the experimental parameters that maximize the weighted sum of the covariance matrix (information matrix). The experimental parameter settings are denoted using E _i =( _xi , _yi ), where _xi is the number of instances and _yi is the input data scale. Let M denote the total number of experimental parameter settings obtained by enumerating all possible scales and instance numbers. Then, using E _i , we can compute a K-dimensional feature vector F _i , where each term corresponds to a term in the predictive model. In this way, we are able to obtain M eigenvectors for all experimental settings. According to D-optimality, when choosing experimental parameters, we choose to maximize the covariance matrix (information matrix) The experimental parameters of the weighted sum, i.e. The constraints are 0≤α _i ≤1,i∈[1,M], where α _i represents the probability of choosing i experimental setting. We represent the total cost of the experiment by adding a budget constraint term B, where y _i / _xi is the cost of running the experiment E _i according to the pricing model on the cloud platform. In solving the above optimization problem, the M experimental settings are sorted and selected according to the probability α _i in a non-increasing order.

Experiment Execution: Once the experimental setup has been chosen, it is necessary to determine which data samples from the entire input dataset will be used to make up the experimental dataset to satisfy the specified scale. The present invention uses random sampling to select data samples from the entire input data set, because random sampling can avoid falling into isolated areas of the data set. After obtaining a small dataset, deploy the specified number of instances using the chosen experiment settings and start running the task, after which the experimental parameters and task completion time are used as training data for building the predictive model.

The specific implementation process of the embodiment is described as follows:

The large-scale data analysis and processing engines used in the embodiment are Spark and Hadoop. On Spark, we ran 3 types of SparkML-based machine learning: classification, regression, and clustering. The classification algorithm uses the text classification benchmark dataset rcv1 with 44,000 features, and the regression and clustering algorithms use 1 million synthetic datasets with 44,000 features. On Hadoop, TeraSort algorithm and WordCount algorithm were run respectively. The TeraSort algorithm is a general-purpose benchmarking application for large-scale data analysis. Its main job is to sort randomly generated records, using a dataset of 200 million samples. The WordCount algorithm is used to calculate the data from Wikipedia articles. Word frequency in 55 million entries.

In AWS's pool of EC2 instance types, choose m4.large (general purpose), c5.large (compute-optimized), r4.large (memory-optimized), and i3.large (storage-optimized), each with 2 vCPUs , and pre-installed Linux system. The data analysis and processing engine models used in the experiment are Apache Spark 2.2 and Hadoop 2.8 respectively. Table 1 lists the configurations and prices for each type of instance.

Table 1

instance type Memory (GiB) instance hard disk Price (USD/hour) m4.large 8 EBS 0.1 c5.large 4 EBS 0.085 r4.large 15.25 EBS 0.133 i3.large 15.25 SSD 0.156

First set the data scale for modeling experiments to 1% to 8%, and the experimental cluster size is limited to 1 to 8 instances. In the embodiment, the experiments with the top 10 probability α _i are used for testing. When selecting input data samples, a starting seed sample is randomly selected from the input data set; then, at each sampling step, output samples are randomly obtained; the above process is repeated until the number of selected samples meets the scale requirements in the experimental parameters. In the embodiment, m4.large is used as the basic instance type, so finally the randomly sampled data set is run on the m4.large cluster of the scale specified in the experimental parameters, and the running time is recorded.

Step 2: Model construction stage, the implementation is as follows,

Model Builder consists of Model Builder and Model Converter. Using the training data collected for a specific instance type, the model builder can build a base predictive model. After that, the model transformer transforms and derives the prediction models of the remaining instance types according to the base prediction model.

Model Builder: When running experiments on a subset of the input dataset on a specific instance type, use T _base (x,y) to represent the task runtime, given the number of instances x and the scale of the dataset y. Large-scale analysis tasks are usually run in successive steps (i.e. iterations) until a termination condition is met. Each step mainly consists of two phases: concurrent computation and data communication. The computational time of task execution remains relative to the dataset size, and there are several representative communication patterns in large-scale analysis tasks. Therefore, the running time of large-scale analysis tasks can be inferred by analyzing the computation time and communication time. The main goal of this embodiment is to obtain the performance prediction function T _base (x, y) for a given task by computing and communicating modes of the task and designing fitting terms involving x and y.

Computational time-consuming, the time cost incurred by the user-defined iterative algorithm operating on each sample of the input data. For large-scale data processing tasks in a cluster computing environment, the computation time can be approximated by several different fitting terms, depending on the characteristics of the dataset (eg, dense or sparse) and the algorithm. Thus, the computation time will be a function of the number of instances and the size of the dataset. Determining the exact fit of a function requires specific domain knowledge.

Communication is time-consuming, and the time cost of data transmission to the target node through the network. Figure 1 abstracts representative communication patterns in large-scale data analysis tasks. Although there are differences in programming models and execution mechanisms, common communication patterns represent most communication situations in clustered applications. The communication time is mainly a function of the number of instances, and the fitting term of the function can be inferred according to the different communication modes of the task. For example, when the data size of each instance is constant, the communication time increases linearly with the number of instances in the partition-aggregate communication mode, but it is a quadratic relationship for the shuffle communication mode.

Given all the candidate fit terms of the function T _base (x,y), we use mutual information as the selection criterion for the fit terms, exclude redundant terms and select only good predictors as the fit terms. Assume represents the set of all candidates, where each term fk is a function of _x and y determined by the computation and communication mode. Given m training data samples collected at different numbers of instances and different data scales, first compute the K-dimensional feature vector F _i = (f _1,i ,...,f _K,i ) for each experimental setting, e.g. f _k,i =y _i / _xi . Then, we compute the mutual information between each item and the running time, and select the items whose mutual information with the running time is higher than a threshold. According to m training runtime samples, the basic prediction model is obtained by fitting The value of w _k in . Wherein β _k indicates whether the fitting item f _k is selected (β _k =1 indicates that this item is selected).

Model converters: Cloud providers typically offer various instance families with different combinations of CPU, memory, hard disk, and network capacity to meet the needs of different jobs, such as general purpose and compute/memory/storage optimized. Through extensive experiments, we found that given a task and a fixed dataset, the runtime of one instance type can be converted to a different instance type according to a simple mapping. Therefore, there is no need to perform experiments on each instance type to obtain training data before building a predictive model, which greatly reduces training time and training cost.

The transformer Φ is the mapping from the base prediction model to the target prediction model Φ:T _base (x,y)→T _target (x,y). By comparing the running times of different instance types under the same task and dataset size, it can be seen that the categories of fit items in the prediction function are similar. In other words, under the same task and dataset size, if f _k is contained in T _base (x, y), it is likely that T _target (x, y) should also contain f _k . This is mainly because the computation and communication patterns of tasks remain largely unchanged under the same application configuration and number of instances. However, under different instance types, the weight of each item will be different, so we need to focus on the weight mapping from the base prediction model to the target prediction model. We adopted a simple and efficient mapping method. Assume represents the lowest cost of the experimental settings chosen by the training data collector, with a running time of t _base . We run experiments on the target instance type to get the running time t _target . The model transformer exports the prediction model of the target instance type as in

The specific implementation of embodiment is as follows:

In the embodiment, the fitting terms added to the prediction function are: constant term, y/x linear term, the square root of the data scale and the number of instances item. A fixed constant representing the time spent in serial computation; for algorithms whose computation time is linear with the size of the dataset, add a fitting term for the scale of the data and the number of instances y/x; for sparse datasets, add the square root of the scale of the data with the number of instances fitting term.

Table 2

communication mode structure fit term Parallel read/write Many One-to-One x Partition-aggregate Many-to-One logx Broadcast One-to-Many x Collect Many-to-One x Shuffle Many-to-Many x² Global communication All-to-All x²

In the embodiment, according to the communication modes of different tasks, the communication fitting items shown in Table 2 are used, which are respectively x, logx, and x ² . After all terms are selected, the base predictive model is computed using a non-negative least squares (NNLS) solver. After that, choose the lowest-cost experiment setting in the base experiment, and run the task on the target instance type with the same experiment setting. Finally, the prediction model for all instance types is derived as

Step 3: Selector construction stage, the implementation is as follows,

Integrate runtime prediction models for all instance types into a single runtime predictor T(x,y), where x is a cloud configuration vector consisting of the type and number of instances. For a given input dataset of tasks, the goal is to enable the user to find the most preferred cloud configuration that satisfies certain runtime and cost constraints. Let P(x) be the price per unit time of cloud configuration x, that is, the unit price of the instance type multiplied by the number of instances. The optimal cloud configuration selection problem can be formulated as x ^* =S(T(x,y),C(x),Ry, where Cx=Px×Tx,y,0≤y≤1

where C(x) is the price per unit time of cloud configuration x and R(y) is a user-added constraint such as the maximum tolerable runtime or the maximum tolerable cost. The selector S(*) is determined by the user and is used to select the best cloud configuration x ^* that satisfies the desired performance or cost.

The specific implementation process of the embodiment is described as follows:

After obtaining all prediction models for all tasks on the instance type to be selected, it is necessary to find the optimal cloud configuration solution with the lowest operating cost. What the cloud configuration solution needs to meet is to enable the task to be completed in the shortest time under the premise of a given cost budget. For the embodiment, the algorithm is evaluated through four test results, namely: effectiveness, prediction accuracy, training cost and application scalability.

Effectiveness: Compare the performance of SILHOUETTE and Ernest on 5 tasks. Figure 3(a) shows that the prediction accuracy of SILHOUETTE is comparable to that of Ernest, and Figure 3(b) shows that the training time and training cost of SILHOUETTE is much lower than that of Ernest. SILHOUETTE can save 25% training time and 30% cost when we build predictive models for 2 instances. As can be seen from Figure 3(c), when there are more candidate instance types, the training time and training cost of SILHOUETTE are much lower than Ernest, while when there are 5 candidate instance types, the training time of SILHOUETTE and Ernest are respectively 25 minutes and 83 minutes. When there are more candidate instance types, it is predictable that SILHOUETTE performs better.

Prediction accuracy: Figures 4 and 5 show that both the prediction accuracy of the m4.large base prediction model and the c5.large transformed prediction model can achieve high accuracy, which confirms the effectiveness of the model transformer in SILHOUETTE.

Training cost: SILHOUETTE aims to find the optimal cloud configuration with low overhead. Therefore, the completion time of the entire task is compared with the training data time for building the underlying predictive model. Figure 6 shows that the training time of SILHOUETTE is less than 20% of the total completion time of all applications except TeraSort.

Application Scalability: On datasets of different sizes, SILHOUETTE uses the same experimental setup to build base and transformed prediction models and evaluate their prediction accuracy. Figure 7 illustrates that the prediction error is consistently below 15% when we use 1.5x, 2x, 2.5x, and 3x the dataset size, which shows that the prediction model built by SILHOUETTE can still maintain a high value even when the size of the dataset changes. accuracy.

In this example, SILHOUETTE is used to select the best cloud configuration for WordCount. Considering the four instance types in Table 1, assume that the selector optimization objective is: given the maximum task completion time, minimize the total cost. Figure 9 shows the total time and total cost of running tasks across the dataset using each instance type. We can observe that the total time for compute-optimized instance type c5.large is comparable to storage-optimized instance type i3.large, and SILHOUETTE will choose the lower cost former.

After that, SILHOUETTE can be used to determine the optimal number of instances for a given instance type. Consider two tasks, TeraSort and WordCount. Figure 9 shows the task running time of the two tasks under different cluster sizes. The running time predicted by SILHOUETTE is very close to the actual running time, so the specific cluster size can be selected.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention pertains can make various modifications or additions to the described specific embodiments or substitute in similar manners, but will not deviate from the spirit of the present invention or go beyond the definitions of the appended claims range.

Claims (6)

1. An efficient cloud configuration selection algorithm for big data analysis tasks, characterized in that it comprises the following steps: Step 1: training data collection: select a certain proportion of input data and the number of cloud server instances used when the task corresponding to the proportion is executed, and determine each group of test parameters and task completion time, wherein the certain proportion refers to the experiment. Use data as a percentage of input data; Step 2: Model construction: Using the test parameters and task completion time in Step 1, with the input data ratio and the number of instances, design a fitting polynomial involving the ratio of input data and the number of instances, and determine the basic prediction model The value of w _k in . Wherein β _k indicates whether the fitting item f _k is selected (β _k =1 indicates that this item is selected); Model conversion: Obtain the running time of the least time-consuming test parameter in step 1 under the target instance type as t _target , and use the mapping method to derive the prediction model of the target instance type as in Step 3: Selector Construction: For a given input dataset for a task, using the predictive model obtained in step 2, compute the most optimal cloud configuration that satisfies specific runtime and cost constraints. 2. the efficient cloud configuration selection algorithm of big data analysis task according to claim 1, is characterized in that: The specific process of selecting a plurality of certain proportions of input data and the number of cloud server instances used in the execution of tasks corresponding to the proportions in the step 1 is as follows: First select a certain range of input data and a certain range of cloud server instances. According to D-optimality, when selecting experimental parameters, choose the maximum covariance matrix (information matrix) The experimental parameters of the weighted sum, i.e. The constraints are 0≤α _i ≤1,i∈[1,M], where α _i represents the probability of choosing i experimental setting, _xi is the number of instances, y _i is the input data ratio, and M represents the total number of experimental parameter settings obtained by enumerating all possible ratios and instances; Express the total cost of the experiment by adding a budget constraint term B, where y _i / _xi is the cost of running the experiment E _i according to the pricing model on the cloud platform; The M experimental settings are sorted in non-increasing order according to the probability α _i , and the experimental parameter group at the top of the ranking is selected as the training data. 3 . The efficient cloud configuration selection algorithm for big data analysis tasks according to claim 1 , wherein the top 10 data groups are selected as training data in the non-increasing order sorting. 4 . 4. the efficient cloud configuration selection algorithm of big data analysis task according to claim 2, is characterized in that: In the step 1, the input data in a certain range is specifically 1% to 10% of the data, and the number of cloud server instances in the certain range is 1-10. 5. the efficient cloud configuration selection algorithm of big data analysis task according to claim 1, is characterized in that: A certain proportion of the input data described in step 1 is selected by random sampling from the entire input data set. 6 . The efficient cloud configuration selection algorithm for big data analysis tasks according to claim 1 , wherein the fitting term in the model construction involves computation time-consuming and communication time-consuming. 7 .