TWI770534B - Automatic machine learning system performance tuning method, device, electronic device and storage medium - Google Patents

Abstract

A method for performance tuning in Automatic Machine Learning (AutoML) includes obtaining a preset application interface and system resources of the automatic machine learning system. Performance index measurement values are obtained according to the preset application interface when system pre-trains deep learning training model candidates. A distribution strategy and a resource allocation strategy are determined according to the performance index measurement values and the system resources and computing resources of the system are allocated according to the distribution strategy and the resource allocation strategy, so that the deep learning training model candidates is trained based on the computing resources, so as to realize the dynamic allocation of computing resources for each deep learning training model candidate, and ensure the rational allocation of resources of the automatic machine learning system, and improve training performance. An electronic device and a storage medium are also disclosed.

Description

Automatic machine learning system performance tuning method, device, equipment and medium

The present invention relates to an automatic machine learning system performance tuning method, device, equipment and medium.

Automated machine learning (Automated Machine Learning, AutoML) technology is one of the hot research and rapid development directions in the field of machine learning. It is a way of combining automation and machine learning. The steps of preprocessing, feature selection, algorithm selection, etc. are combined with the steps of model architecture design and model training in deep learning, and they are placed in a "black box". Through the black box, users only need to input data, they can get its desired prediction result. Many companies at home and abroad have integrated AutoML technology into their self-developed AI platforms, reducing the cost of parameter adjustment, trial and error for algorithm engineers, and accelerating the construction and implementation of machine learning models. The existing AutoML platform products include: Cloud AutoML, EasyDL, Cloud PAI, DarwinML, AI Prophet AutoML, and Zhiyi Technology.

Automated machine learning realizes automation from three aspects: feature engineering, model building, and hyperparameter optimization. Automated machine learning can be divided into two categories. When the supported model category is classification or regression, the techniques used include probabilistic matrix factorization and Bayesian optimization, which require less computation and therefore lower implementation costs. Another class of models supported are Convolutional Neural Networks (CNN), Cyclic Neural Networks for classification (RNN), Long Short-Term Memory Network (LSTM), using techniques including reinforcement learning with gradient policy updates, efficient neural architecture search, which uses an RNN controller trained through loops to This candidate architecture is sampled and then trained to measure its performance on the desired task, and the controller then uses the performance as a guiding signal to find more promising architectures. Neural architecture search is computationally expensive and time-consuming.

In summary, when using AutoML for deep learning, the process of developing a neural network needs to consume a lot of computing power, and the computing resources requested for each randomly selected candidate architecture are different, and there is a problem of over-allocation or under-allocation of computing resources. .

In view of the above, it is necessary to provide an automatic machine learning system performance tuning method, device, equipment and medium, which can dynamically allocate the computing resources of the deep learning training model in the automatic machine learning, and solve the problem of excessive allocation or allocation of computing resources. Insufficient problem, improve automatic machine learning training performance.

An embodiment of the present application provides a performance tuning method for an automatic machine learning system, which is applied to a performance tuning device connected to the automatic machine learning system, including: acquiring a default application programming interface and system resources of the automatic machine learning system ; When the automatic machine learning system pre-trains a candidate deep learning training model, obtain its corresponding performance index measurement value according to the preset application program interface; According to the performance index measurement value and the system Resource determination distribution strategy and/or resource allocation strategy; and allocating computing resources of the automatic machine learning system according to the distribution strategy and/or the resource allocation strategy, so that the candidate deep learning training model is based on the calculation Resource allocation for training.

An embodiment of the present application further provides a performance tuning device, including: a first acquisition module for acquiring a preset application program interface and system resources of the automatic machine learning system; a second acquisition module for When the automatic machine learning system pre-trains a candidate deep learning training model, the corresponding performance index measurement value is obtained according to the preset application program interface; the strategy determination module is used for measuring the value according to the performance index and the system resources to determine a distribution strategy and/or a resource allocation strategy; and an allocation module for allocating computing resources of the automatic machine learning system based on the distribution strategy and/or the resource allocation strategy, so that all The candidate deep learning training model is trained based on the computing resource configuration.

An embodiment of the present application also provides an electronic device, the electronic device includes: one or more processors; when one or more programs are executed by the one or more processors, the one or more processors implement any one of the above The described automatic machine learning system performance tuning method.

An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the performance tuning method for an automatic machine learning system as described in any one of the above.

In the automatic machine learning system performance tuning method, device, device and medium provided by the embodiments of the present application, each candidate deep learning training model in the automatic machine learning system is independently dynamically optimized. For each candidate deep learning training model in the system, obtain the performance index measurement value of the preset application program interface when the candidate deep learning training model is pre-trained, and determine the distribution strategy according to the performance index measurement value and the system resources and/or resource allocation strategy, and finally assign the automatic machine learning system based on the distribution strategy and/or the resource allocation strategy Systematic computing resources, so that the candidate deep learning training model is trained based on the computing resource configuration, realizes the dynamic allocation of computing resources for each candidate deep learning training model, ensures the reasonable configuration of computing resources of the automatic machine learning system, and improves the training performance.

10: Automated Machine Learning Systems

73: Performance Tuning Device

7: Electronic equipment

71: Processor

72: Memory

101: The first acquisition module

102: Second acquisition module

103: Strategy Determination Module

104: Assign modules

FIG. 1 is a schematic diagram of performance tuning of an automatic machine learning system provided by an embodiment of the present application.

FIG. 2 is a block diagram of an electronic device provided by an embodiment of the present application.

FIG. 3 is a flowchart of an automatic machine learning system performance tuning method provided by an embodiment of the present application.

FIG. 4 is a schematic flowchart of a default application programming interface and a method for obtaining system resources provided by an embodiment of the present application.

FIG. 5 is a schematic flowchart of a method for configuring an API according to an embodiment of the present application.

FIG. 6 is a block diagram of an apparatus for performance tuning provided by an embodiment of the present application.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

In order to facilitate the in-depth understanding of the embodiments of the present application by those skilled in the art, the following will firstly introduce definitions of technical terms involved in the embodiments of the present application.

In the process of developing a neural network, AutoML will upload the training set, and search for the best neural network architecture through Neural Architecture Search (NAS, NeuralArchitecture Search via Parameter Sharing). Start with "building blocks" that may be used by the neural network, these "building blocks" include various convolution and pooling blocks, and then use a recurrent neural network (RNN) as the controller, the controller from Pick from these "building modules", then put them together to form a new neural network architecture, use the training set to train the new neural network architecture until convergence, use the test set to test, and get the accuracy rate, this standard The accuracy rate is then used to update the controller through policy gradients, allowing the controller to generate higher and higher levels of neural network architecture. The methods of learning convolutional neural network structure also include Efficient Neural Architecture Search via Parameter Sharing (ENAS) and Progressive Neural Architecture Search.

Referring to FIG. 1 , in order to optimize the performance of the automatic machine learning system 10 and reduce the training time of the automatic machine learning system 10 , the performance tuning device 73 is connected to the automatic machine learning system 10 to perform performance tuning through the embodiment of the present application. The device 73 automatically optimizes the automatic machine learning performance of the automatic machine learning system 10 for each candidate depth randomly selected by the automatic machine learning system 10 Learning and training models allocate computing resources to avoid over-allocation or under-allocation of computing resources for each candidate deep learning training model.

In the embodiment of the present application, the AutoM1 bottom layer of the automatic machine learning system 10 can use tools such as Scikit-Learn, XGBoost, TensorFlow, Keras, LightGBM, etc., to ensure the efficiency of the runtime.

Please refer to FIG. 2 , the performance tuning device 73 can run on an electronic device. The electronic device includes, but is not limited to, a memory and at least one processor. The above components can be connected through a bus, and the performance tuning device 73 runs on the processor, and the performance tuning device 73 implements the steps in the embodiment of the automatic machine learning system performance tuning method of the present application when the performance tuning device 73 executes the computer program. Alternatively, when the performance tuning device 73 executes the computer program, the functions of each module/unit in the embodiment of the performance tuning device 73 of the present application are implemented.

In this embodiment, the electronic device may include a performance tuning device 73 and a server. In other embodiments, the electronic device may be a computing device such as a cloud server. Those skilled in the art can understand that the schematic diagram is only an example of an electronic device, and does not constitute a limitation on the electronic device, and may include more or less components than the one shown in the figure, or combine some components, or different components, For example, the electronic device may further include input and output devices, network access devices, bus bars, and the like. The automatic machine learning system performance tuning method of the present application is applied in one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs) , Programmable Gate Array (Field-Programmable Gate Array, FPGA), Digital Signal Processor (Digital Signal Processor, DSP), embedded devices, etc.

The electronic device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a cloud server. The electronic device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice-activated device.

FIG. 3 is a flowchart of an automatic machine learning system performance tuning method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Please refer to FIG. 3, the automatic machine learning system performance tuning method is implemented by the performance tuning device 73 connected to the automatic machine learning system 10, and specifically includes the following steps:

Step S10: Acquire the default application programming interface and system resources of the automatic machine learning system.

In this embodiment of the present application, the performance tuning device 73 is connected to the automatic machine learning system 10 , and the computing backend of the automatic machine learning system 10 is a computing backend that the performance tuning device 73 can identify, such as TensorFlow, the performance tuning device 73 can obtain relevant data information of the automatic machine learning system 10 , and obtain the default application programming interface and system resources of the automatic machine learning system 10 .

In this embodiment of the present application, the performance tuning device 73 may include an tuning server and a performance analysis tool.

In one of the possible implementations, please refer to FIG. 4 , the acquisition of the default application programming interface and system resources of the automatic machine learning system 10 can be specifically performed through the following steps:

Step S101 : Record the application programming interface related to the system performance in the automatic machine learning system and the system resources of the automatic machine learning system in a database of a tuning server.

Step S102: the tuning server reads the system resources of the automatic machine learning system from the database.

Step S103: A performance analysis tool reads the default API from the database of the tuning server.

In the embodiment of the present application, both the tuning server and the performance analysis tool in the performance tuning device 73 can identify the computing backend of the automatic machine learning system 10, and the tuning server pre- Record performance-related Application Programming Interface (API) and system available resources in the automatic machine learning system 10 in its database, wherein the default API is the automatic machine learning system 10 APIs related to performance in performing deep learning task stacks.

In this embodiment of the present application, the tuning server may be a built-in server that generates the distribution strategy and resource configuration strategy.

In the embodiment of the present application, the performance analysis tool may be a SOFA server with a built-in efficiency measurement tool, and may further include a flame graph (Flame Grap) with a built-in efficiency measurement tool. Through the performance analysis tool, heterogeneous performance index measurement values can be collected from a central processing unit (CPU), a graphics processing unit (GPU), a communication network, and a storage device.

Step S20: When the automatic machine learning system pre-trains the candidate deep learning training model, obtain the corresponding performance index measurement value according to the preset application program interface.

In the embodiment of the present application, after the user logs the training data into the automatic machine learning system 10, the automatic machine learning system 10 performs a neural network search, and searches for the candidate deep learning training model according to the training data. Pre-training is performed, and during the deep learning training process, the performance index measurement values corresponding to the application program interface are obtained according to the preset application program interface.

In one possible implementation manner, when the automatic machine learning system 10 pre-trains a candidate deep learning training model, acquiring the corresponding performance index measurement value according to the preset application program interface includes: When the automatic machine learning system 10 pre-trains a candidate deep learning training model, the performance analysis tool obtains the performance index measurement value of the application program interface according to the default application program interface, and uses a communication method such as a remote The terminal calls GRPC to transmit the measurement value of the performance index Input to the tuning server, wherein the performance analysis tool is integrated and packaged with the automatic machine learning system 10 .

In the embodiment of the present application, the performance analysis tool is integrated and packaged with the automatic machine learning system 10, and by integrating the performance analysis tool, the automatic machine learning model and the corresponding application programming interface software package, the automatic acquisition of the automatic The machine learning system 10 measures the performance index value, and sends the performance index measurement value to the built-in server through a remote call, so that the built-in server makes a computing resource decision.

Illustratively, AutoKeras is selected as the automatic machine learning system 10 engine, which will use an efficient neural architecture search algorithm (ENAS) to select and evaluate the candidate deep learning models to The evaluation of the learned model selects the next better candidate deep learning model. The user logs the training data into the AutoKeras, the data preprocessing API of the AutoKeras determines the amount of data for one training according to the hyperparameter batch size of the deep neural network, and the candidate deep learning model is trained according to the training data In the process, the SOFA server obtains the performance index measurement value of the default API of the AutoKeras through the performance measurement tool, for example, obtains the high-speed serial computer expansion bus standard bus through the performance measurement tool A measure of the number of data exchanges on the row (PCIe BUS). The SOFA server sends the obtained performance index measurement value to the built-in server through remote program calls (such as gRPC Remote Procedure Calls, etc.), so that the built-in server can be the candidate according to the performance index measurement value and system resources. Deep learning models allocate computing resources.

In one of the possible implementations, the performance metrics include forward propagation time, (FW) and backward propagation time (BW), time to copy data from the host to the GPU device (H2D), copy data from the GPU device to the host (D2H) or when data is copied from peer-to-peer (P2P).

In one possible implementation manner, the performance indicators include program execution time in user space and system space, read/write bandwidth of the file system or storage medium, network usage bandwidth, and call Instantaneous hotspot distribution, time-consuming system lock overhead, etc.

Step S30: Determine a distribution strategy and/or a resource allocation strategy according to the performance index measurement value and the system resources.

In the embodiment of the present application, the performance analyzer sends the performance index measurement value to the built-in server through a remote program call, and the built-in server determines the performance index measurement value according to the performance index measurement value and the system resources Distribution strategy and/or resource allocation strategy.

In one of the possible implementations, the built-in server will create another markup language (YAML Ain't Markup Language, YAML) file for generating a Kubernetes pod, where a pod is the smallest unit that can be created and deployed in Kubernetes , that is, an application instance in a Kubernetes cluster. Kubernetes is used to manage containerized applications on multiple hosts in a cloud platform. It is an open source platform for automating container operations. The YAML file will record the specific application container engine (Docker) mapping to run, the allocation of hardware resources, and the settings of the virtual machine node of the corresponding container.

In one possible implementation manner, the determining the distribution strategy according to the performance index measurement value and the system resources includes: determining single-node training or multi-node training according to the performance index measurement value and the system resources, Wherein, the single-node training includes training the candidate deep learning training model by a single node, and the multi-node training includes jointly training the candidate deep learning training model by a plurality of nodes, and sharing the candidate deep learning training model Parametric variables in the model.

In the embodiment of the present application, the training task for the candidate deep learning training model can be determined according to the measured value of the performance index and the system resource to be trained by a single node or by multiple nodes. , through the distributed training distribution among the complex nodes, allowing the deep learning training task to be scaled up to learn a larger model or train at a faster speed practice. When the training task is light, training through a single node can ensure the reasonable allocation of computing resources and avoid excessive allocation of computing resources.

In one possible implementation manner, the single-node training includes training the candidate deep learning training model by a single device or a plurality of mirrored devices in a single node. The multi-node training includes training the candidate deep learning training model by a plurality of nodes using a replication mode or a parameter server mode.

In this embodiment of the present application, when training is performed by a single node, the training may be performed by a single device, for example, by a single graphics processor, and parameters are stored by a graphics processor or a central processing unit. The candidate deep learning training model may also be trained by a plurality of mirroring devices, for example, a graphics processor of a complex mirroring device trains the candidate deep learning training model, and then the graphics processor stores parameters.

In the embodiment of the present application, joint training is performed on multiple nodes, that is, the training of the candidate deep learning training model is performed in a distributed manner, the candidate deep learning training model is jointly trained by multiple programs synchronously, and the candidate deep learning training model is shared. Parametric variables of the model, such as weights, bias values, etc. Illustratively, when using replication mode, training is performed by a complex graphics processor on a complex node, and parameters are stored by the graphics processor. When the parameter server mode is used, the parameters of the candidate training model are stored separately from the training data based on the parameter server (Parameter-Server), the training is performed by the graphics processor, and the parameters are stored by the central processing unit.

In one of the possible implementations, when using the TensorFlow API for deep learning distributed training, if the number of high-speed serial computer expansion bus (PCIe BUS) data exchanges obtained by the SOFA server is small, the POTATO It is recommended to use a parameter server mode, such as Parameter Server, to support the distributed storage and collaboration of large-scale parameters; on the contrary, use a replication mode such as Mirrored Replicated to transfer a regional data center (Data Center) through the network. Center) is transferred to the data center in the target region.

In one possible implementation manner, the determining a resource allocation policy according to the performance index measurement value and the system resources includes: training the candidate deep learning model for the candidate deep learning model according to the performance index measurement value and the system resources Configure the API, software resources, and hardware resources.

In the embodiment of the present application, the built-in server determines the application programming interface, matching software resources and hardware resources used by the candidate deep learning model currently selected by the automatic machine learning system 10 .

In one of the possible implementations, please refer to FIG. 5 , the configuration of the application program interface for the candidate deep learning training model according to the performance index measurement value and the system resource can be specifically performed through the following steps:

Step S301: Determine the API type of the candidate deep learning training model.

In the embodiment of the present application, the API type required by the candidate deep learning training model is re-determined.

Step S302: Determine a new API of the candidate deep learning training model according to the API type.

In the embodiment of the present application, a new API is allocated to the candidate deep learning training model according to the re-determined API type.

Step S303: Adjust the parameters of the new API by sharing the environment variables of the automatic machine learning system, wherein the parameters include batch size.

In the embodiment of the present application, the built-in server adjusts the new application programming interface by sharing the environment variables of the automatic machine learning system 10, and re-executes the new application programming interface. Illustratively, determine the affordable deep learning model based on the computing power of the GPU and its memory size The maximum value for batch processing and is set in the new API so that when the API is restarted, the data size of a training session of the candidate deep learning model can be adjusted.

In one of the possible implementations, the same candidate depth model is shared among all connected computing nodes by using a remote calling GRPC to perform batch data parallel computing.

Step S40: Allocate computing resources of the automatic machine learning system according to the distribution strategy and/or the resource configuration strategy, so that the candidate deep learning training model is trained based on the computing resource configuration.

In the embodiment of the present application, the candidate deep learning training model is trained based on the computing resource configuration, and the automatic machine learning system 10 evaluates the performance of the candidate deep learning training model, and then continues to select according to the model evaluation result Model training is performed on the new candidate deep learning training model, and steps S10 to S40 are continued to allocate computing resources to the new candidate deep learning training model, until a candidate deep learning training model that meets the requirements is obtained.

In this embodiment of the present application, the built-in server sends the determined distribution strategy and/or the resource allocation strategy to the automatic machine learning system 10, and allocates the determined distribution strategy and/or the resource allocation strategy to the automatic machine learning system 10. computing resources of the automatic machine learning system 10, so that the automatic machine learning system 10 configures computing resources for the currently selected candidate deep learning training model according to the allocated computing resources, such as the number of allocated CPU cores, the capacity of the main memory , the number of GPUs, etc., the candidate deep learning training model is trained based on the computing resource configuration.

When trying new candidate deep learning models in each round, the automatic machine learning system 10 dynamically optimizes performance according to the characteristics of each newly selected candidate deep learning model through the automatic machine learning system performance tuning method.

In one possible implementation manner, after the automatic machine learning system 10 is optimized, a corresponding automatic machine learning application program interface is generated, the test data is logged into the automatic machine learning application program interface, and the automatic machine learning application program interface is recorded. test data.

Referring to FIG. 6 , an embodiment of the present application provides a performance tuning device 73 , which includes a first acquisition module 101 , a second acquisition module 102 , a policy determination module 103 and an allocation module 104 .

The first obtaining module 101 is used for obtaining the default application programming interface and system resources of the automatic machine learning system.

The second obtaining module 102 is configured to obtain a corresponding performance index measurement value according to the preset application program interface when the automatic machine learning system pre-trains a candidate deep learning training model.

The strategy determination module 103 is configured to determine a distribution strategy and/or a resource allocation strategy according to the performance index measurement value and the system resources.

The allocation module 104 is configured to allocate computing resources of the automatic machine learning system according to the distribution strategy and/or the resource allocation strategy, so that the candidate deep learning training model is trained based on the computing resource allocation.

The processor 71 may be a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), or an application specific integrated circuit (ASIC). ), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic elements, discrete gate or transistor logic elements, discrete hardware elements, and the like. The general-purpose processor can be a microprocessor or the processor 71 can also be any conventional processor, etc. The processor 71 is the control center of the electronic device 7, and uses various interfaces and lines to connect the entire electronic device 7. various parts.

The memory 72 can be used to store the computer programs and/or modules/units, and the processor 71 executes or executes the computer programs and/or modules/units stored in the memory 72, and calls The data stored in the memory 72 realizes various functions of the electronic device 7 . The memory 72 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; The data storage area can store data or the like created according to the use of the electronic device 7 . In addition, the memory 72 may include high-speed random access memory, and may also include non-volatile memory such as hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (Secure Digital, SD) card, flash memory card (Flash Card), at least one disk memory device, flash memory device, or other volatile solid state memory device.

If the modules/units integrated in the electronic device 7 are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, so When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-only memory) Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, electrical signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium Electric carrier signals and telecommunication signals are not included.

In the several embodiments provided by the present invention, it should be understood that the disclosed electronic device and method may be implemented in other manners. For example, the above-described electronic device embodiments are only illustrative. For example, the division of the modules is only a logical function division, and other division methods may be used in actual implementation.

In addition, each functional module in each embodiment of the present invention may be integrated in the same processing module, or each module may exist physically alone, or two or more modules may be integrated in the same processing module. in the same module. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present invention.

7: Electronic equipment

71: Processor

72: Memory

73: Performance Tuning Device

Claims (10)

An automatic machine learning system performance tuning method, which is applied to an performance tuning device connected to the automatic machine learning system, wherein the automatic machine learning system performance tuning method comprises: obtaining a preset of the automatic machine learning system Application programming interface and system resources; when the automatic machine learning system pre-trains the candidate deep learning training model, obtains its corresponding performance index measurement value according to the default application programming interface; according to the performance index measurement value and the system resources to determine a distribution strategy and/or a resource allocation strategy; and, allocating computing resources of the automatic machine learning system according to the distribution strategy and/or the resource allocation strategy, so that the candidate deep learning training The model is trained based on the computing resource configuration. The automatic machine learning system performance tuning method according to claim 1, wherein the determining a distribution strategy according to the performance index measurement value and the system resource comprises: according to the performance index measurement value and the system resource Determine single-node training or multi-node training, wherein the single-node training includes training the candidate deep learning training model by a single node, and the multi-node training includes jointly training the candidate deep learning training model by a plurality of nodes , and share the parameter variables in the candidate deep learning training model. The automatic machine learning system performance tuning method according to claim 1, wherein the single-node training includes training the candidate deep learning training model by a single device or a plurality of mirrored devices in a single node; the multi-node training This includes training the candidate deep learning training model by the plurality of nodes using a replication mode or a parameter server mode. The automatic machine learning system performance tuning method according to claim 3, wherein, The determining a resource allocation strategy according to the performance index measurement value and the system resource includes: configuring an application programming interface, software resources, and hardware resources. The automatic machine learning system performance tuning method of claim 4, wherein the configuring an application program interface for the candidate deep learning training model according to the performance index measurements and the system resources comprises: determining the candidate an API type for a deep learning training model; determine a new API for the candidate deep learning training model according to the API type; adjust the new application by sharing environmental variables of the automatic machine learning system Interface parameters, wherein the parameters include batch size. The performance tuning method for an automatic machine learning system according to claim 1, wherein the acquiring the default application programming interface and system resources of the automatic machine learning system comprises: Application programming interface and system resources of the automatic machine learning system are recorded in a database of a tuning server; the tuning server reads the system resources of the automatic machine learning system from the database; a performance analysis The tool reads the default API from the tuning server's database. The performance tuning method for an automatic machine learning system according to claim 1, wherein, when the automatic machine learning system pre-trains a candidate deep learning training model, the corresponding information is obtained according to the default application program interface. The performance index measurement value includes: when the automatic machine learning system pre-trains a candidate deep learning training model, the performance analysis tool obtains the performance of the application program interface according to the default application program interface The index measurement value is transmitted to the tuning server by means of communication, wherein the performance analysis tool is integrated and packaged with the automatic machine learning system. A performance tuning device, wherein the performance tuning device comprises: a first acquisition module for acquiring a preset application program interface and system resources of the automatic machine learning system; a second acquisition module for When the automatic machine learning system pre-trains the candidate deep learning training model, the corresponding performance index measurement value is obtained according to the preset application program interface; the strategy determination module is used for measuring the value according to the performance index and the system resource to determine a distribution strategy and/or a resource allocation strategy; and an allocation module for allocating computing resources of the automatic machine learning system according to the distribution strategy and/or the resource allocation strategy, so that the The candidate deep learning training model is trained based on the computing resource configuration. An electronic device, wherein the electronic device comprises: one or more processors; when one or more programs are executed by the one or more processors, so that the one or more processors implement any one of claim 1 to 7 The automatic machine learning system performance tuning method described in item. A computer-readable storage medium, wherein a computer program is stored, and when the computer program is executed by a processor, the automatic machine learning system performance tuning method according to any one of claim 1 to 7 is implemented.

Images