KR20200127467A - Management system and method of machine learning platform - Google Patents

Abstract

According to an embodiment of the present invention, provided is a machine learning platform management system capable of managing various machine learning related frameworks with one platform. The machine learning platform management system comprises: a master node that manages machine learning tasks and commands registered through an external program; a session node that analyzes machine learning task information and the commands received from the master node and assigns the same to an executable lower node; and a worker node that performs machine learning according to the machine learning task information and the commands assigned from the session node, and reports learning progress and resource status to the session node.

Description

Machine learning platform management system and method {MANAGEMENT SYSTEM AND METHOD OF MACHINE LEARNING PLATFORM}

An embodiment of the present invention relates to a machine learning platform management system and method, and more particularly, to a machine learning platform management system and method that can be applied to a machine learning environment to which various frameworks are applied.

Recently, global IT companies such as Google and Microsoft and academic research groups have released platforms related to machine learning technology (framework, library, etc.) for free. The unveiled machine learning platform provides an interface that users can easily use even if they do not know about complex machine learning internal technologies, dramatically lowering the barriers to entry for users to use the technology. Companies or research groups that have released open platforms are constantly updating new or improved learning algorithms and advanced hardware utilization technologies through Github and its web, and most open platforms are available through intuitive user interfaces. It was developed with the aim of improving the performance of learning models using CPU and GPU, and scalability based on distributed platforms.

In November 2015, the Google Brain Team released TensorFlow, an improved version of the existing deep learning platform DistBelief, to derive flexible and scalable results, and express the results in an intuitive graph during the learning process. At the developer conference, TensorFlow Lite, developed to utilize machine learning on the Android mobile platform, was unveiled.

Mahout, a sub-project of the Apache Software Foundation's Lusin project, was developed with the aim of implementing an extensible machine learning library, and in 2010, it was converted to a top-level Apache project stage. Hadoop file system, which is a large-capacity data processing system, is used, and the Spark system with improved performance and user convenience is used.

MLlib, an internal project of the Apache Software Foundation's Spark Project, was developed to provide high-speed machine learning technology that utilizes large amounts of data, and was released in 2013. It is known to be faster than disk-based Mahout because it operates based on memory, but the deep learning technology it supports is limited.

Caffe2, which was unveiled by Facebook's artificial intelligence research group in July 2017, is a deep learning platform based on the existing Caffe. Researched and developed, and focused on improving the performance, portability, and usability of learning models.

In recent years, the necessity of machine learning has increased, and at the same time, companies and academia have unveiled various machine learning platforms, and if the environment is applicable to them, even users who do not know specific technologies can develop machine learning-based applications.

However, in order to use a machine learning platform, you need to configure a machine learning environment. For example, to use TensorFlow you usually need the Python language. It goes through the complicated process of building a virtual environment like Anaconda to install the Python language, or installing TensorFlow and an IDE like PyCharm to work with TensorFlow. In the process of configuring such an environment, general users encounter various problems such as dependency problems and version compatibility problems, so the barrier to entry is high in building an environment to perform machine learning.

An object of the present invention is to provide a machine learning platform management system and method capable of managing various machine learning related frameworks as a single platform.

In addition, it is to provide a machine learning platform management system and method capable of distributing machine learning tasks to optimal nodes and monitoring execution results in real time.

In addition, it is to provide a machine learning platform management system and method capable of remotely performing machine learning through an external program.

According to an embodiment of the present invention, a master node for managing machine learning tasks and instructions registered through an external program; A session node that analyzes the machine learning task information and the command received from the master node and allocates it to an executable lower node; And a worker node that performs machine learning according to the machine learning task information and the command allocated from the session node, and reports a learning progress status and resource status to the session node.

The session node may analyze computational processing capability according to resource conditions reported from the worker node, and allocate the machine learning task information and the instruction based on the analyzed computational processing capability.

The session node may generate a list of worker nodes sequentially arranged according to the computational processing capability, and allocate the machine learning task information and the command according to the list of worker nodes.

The resource status may include at least one of CPU resource information, GPU resource information, memory resource information, disk resource information, package version, library version, Python language version, framework type, framework version, operating system type, and operating system version. have.

The session node may manage at least one worker node using the same framework.

The worker node may access an external network storage according to the allocated machine learning task information and the command to download the machine learning task and training data files.

The learning progress status may include at least one of a loss value, learning accuracy, validation value, and progress rate.

The command may include at least one of allocation, execution, suspension of the machine learning task, and reporting of a learning progress status.

According to an embodiment of the present invention, the worker node transmits resource information to a session node to request connection; Checking, by the session node, whether the worker node is a node capable of performing machine learning through the resource information; Forming a group by the session node using at least one worker node capable of performing machine learning; Receiving, by the master node, registering machine learning tasks and instructions through an external program; Transmitting, by the master node, the machine learning task information and the command to the session node; Analyzing, by the session node, the machine learning task information and the instruction and allocating it to an executable worker node; Performing, by the worker node, machine learning according to the machine learning task information and the command allocated from the session node; And reporting the learning progress status and the resource status to the session node by the worker node.

In the forming of the group, a group may be formed using at least one worker node using the same framework.

The machine learning platform management system and method according to the present invention can manage various machine learning related frameworks with one platform.

In addition, machine learning tasks can be distributed to optimal nodes and execution results can be monitored in real time.

In addition, machine learning can be performed remotely through an external program.

1 is a conceptual diagram of a machine learning platform management system according to an embodiment.
2 is a block diagram showing a configuration of a master node according to an embodiment
3 is a diagram for describing an operation of a platform management database according to an embodiment.
4 is a block diagram of a configuration of a session node according to an embodiment.
5 is a block diagram of a worker node according to an embodiment.
6 is a flowchart of a method for managing a machine learning platform according to an embodiment.
7 is a diagram illustrating a process of connecting a worker node according to an embodiment.
8 is a diagram for explaining a machine learning task allocation process according to an embodiment.
9 is a diagram for describing a process of executing a machine learning instruction according to an embodiment.
10 is a diagram illustrating an operation of a resource monitoring manager according to an embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

However, the technical idea of the present invention is not limited to some embodiments to be described, but may be implemented in various different forms, and within the scope of the technical idea of the present invention, one or more of the constituent elements may be selectively selected between the embodiments. It can be combined with and substituted for use.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention are generally understood by those of ordinary skill in the art, unless explicitly defined and described. It can be interpreted as a meaning, and terms generally used, such as terms defined in a dictionary, may be interpreted in consideration of the meaning in the context of the related technology.

In addition, terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In the present specification, the singular form may include the plural form unless specifically stated in the phrase, and when described as "at least one (or more than one) of A and (and) B and C", it is combined with A, B, and C. It may contain one or more of all possible combinations.

In addition, terms such as first, second, A, B, (a), and (b) may be used in describing the constituent elements of the embodiment of the present invention.

These terms are only for distinguishing the component from other components, and are not limited to the nature, order, or order of the component by the term.

And, when a component is described as being'connected','coupled' or'connected' to another component, the component is not only directly connected, coupled or connected to the other component, but also the component and It may also include the case of being'connected','coupled' or'connected' due to another component between the other components.

In addition, when it is described as being formed or disposed in the "top (top) or bottom (bottom)" of each component, the top (top) or bottom (bottom) is one as well as when the two components are in direct contact with each other. It also includes a case in which the above other component is formed or disposed between the two components. In addition, when expressed as "upper (upper) or lower (lower)", the meaning of not only an upward direction but also a downward direction based on one component may be included.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings, but the same reference numerals are assigned to the same or corresponding components regardless of the reference numerals, and redundant descriptions thereof will be omitted.

1 is a conceptual diagram of a machine learning platform management system according to an embodiment. Referring to FIG. 1, the machine learning platform management system 1 according to the embodiment may include a master node 10, a session node 20, and a worker node 30. In the embodiment, one master node 10 may manage a plurality of session nodes 20, and one session node 20 may manage a plurality of worker nodes 30. That is, a lower node of the master node 10 may be defined as a session node 20, and a lower node of the session node 20 may be defined as a worker node 30.

The master node 10 may manage machine learning tasks and commands registered through the external program 2. In an embodiment, the external program 2 may be a web client.

2 is a block diagram illustrating a configuration of a master node according to an embodiment. Referring to FIG. 2, the master node 10 may include a resource monitoring manager 11, a task manager 12, a command processor 13, a client manager 14, and a platform management database 15. .

The resource monitoring manager 11 may monitor resources of the worker nodes 30 managed by each connected session node 20. In addition, the resource monitoring manager 11 may check resource conditions that can be used for machine learning in the platform.

The resource situation can be classified into a hardware resource situation and a software resource situation. The hardware resource status may include CPU resource information, GPU resource information, memory resource information, disk resource information, and the like. Specifically, the hardware resource situation can be expressed as GFLOPS, which is a unit of computational processing capability of each CPU and GPU, memory usage and total size, disk usage and total size information. The software resource status may include package version, library version, Python language version, framework type, framework version, operating system type, and operating system version.

The resource monitoring manager 11 can grasp the available resources of each session node 20 and the worker node 30 by grasping the total available resources in the platform. The resource monitoring manager 11 may transmit the resource status to the external program 2 and provide a user to understand the platform resource status in real time.

In addition, the resource monitoring manager distributes the machine learning tasks according to the resource status of the session nodes, and can reconfigure the allocation of worker nodes per session node according to the amount of requests to perform machine learning tasks for each framework.

The task manager 12 may provide a function of registering a learning file and a data set to the platform through the external program 2. The task manager 12 may transmit machine learning task information to the session node 20. Learning files and data sets registered through the external program 2 may be stored in a folder specific to each task in the external network storage 3. The task manager 12 may update and store information about a corresponding unique folder and user information in the platform management database 15. When performing machine learning, the worker node 30 accesses the external network storage 3 based on the task information stored in the platform management database 15, downloads the corresponding task, and performs learning.

The command processor 13 may process a command related to machine learning for a task registered by a user through the external program 2. When a user inputs a command related to machine learning execution through an external program, the command processor 13 checks the corresponding task information in the platform management database 15 and transmits the machine learning task information and commands to the session node 20. have.

The client manager 14 may manage user information registered from the external program 2. User information may include information related to ID, password, and authority. The registered user information may be updated and stored in the platform management database 15.

The platform management database 15 may store platform machine learning tasks, user information, data, and log information of learning execution results.

3 is a diagram for describing an operation of a platform management database according to an embodiment. Referring to FIG. 3, user-related information is managed as a user table, and an Idx item is connected to user_idx of a task table, and can be used to classify a task registered by a user. User Table manages user ID, password, and authority.

Task-related information is managed as a task table, and Idx is connected to Task_idx in the log table, and the log is connected to the log, which is the result value of the task and machine learning, and is used to classify the log of the result information of the task that has been trained. The task table is the user ID of the task, learning status, name, IP and connection port of the assigned session node, IP and connection port of the assigned worker node, process ID when running on the worker node, progress, accuracy of learning, task. Manage information composed of registered time.

The result value of task execution is managed as a log table, and it is connected to the idx of the task and the task_idx of the log table, and can be used to classify the result values of each task. The log table manages the order of result values and outputs of results from machine learning execution.

The Data Table is managed by being connected to the User_idx of the task, and can be used to classify the dataset of each task. The Data Table is the content of data to be machine learning and manages information consisting of the name of the dataset, the user's memo, the registered time, and the path stored in the external network storage.

The session node 20 may analyze machine learning task information and instructions received from the master node 10 and allocate them to an executable lower node.

The session node 20 may analyze operation processing capability according to the resource situation reported from the worker node 30 and allocate machine learning task information and instructions based on the analyzed operation processing capability.

In addition, the session node 20 may generate a list of worker nodes sequentially arranged according to computational processing capability, and allocate machine learning task information and instructions according to the list of worker nodes.

The session node 20 may manage at least one worker node using the same framework.

4 is a block diagram of a configuration of a session node according to an embodiment. Referring to FIG. 4, the session node 20 may include a command analyzer 21 and a worker node manager 22.

The command analyzer 21 may interpret the command for the machine learning task received from the master node 10 and transmit it to a suitable worker node 30. The instruction may include at least one of assigning, executing, stopping, and reporting progress of a machine learning task. The command received from the master node 10 is transmitted to a worker node 30 capable of performing an appropriate task through an analysis process.

The worker node manager 22 may monitor resources of a worker node managed in real time. The worker node manager 22 determines the hardware resource status from each worker node in real time: CPU resource information, GPU resource information, memory resource information, disk resource information, package version, library version, Python language version, framework type, and framework version. , The resource status including at least one of the operating system type and the operating system version may be reported. The worker node manager 22 may generate a list of worker nodes in the order of high computational processing capability (GFLOPS) according to the resource status of the worker node 30 managed based on the resource status reported from each worker node.

The worker node list may mean a list in which the worker node manager 22 sorts worker nodes in order of high computational processing capability based on the monitored resource situation. The worker node manager 22 may update the worker node list by reflecting the status of the worker node whenever the resource status changes according to the period of monitoring the resource status. Through this, the machine learning task given in the platform can be quickly processed in real time by the worker node 30 having the optimum processing capability.

When the worker node manager 22 receives a command to reconfigure a worker node from the master node, the worker node manager 22 transmits connection information of a session node allocated to a worker node to change the session node.

The worker node 30 may perform machine learning according to machine learning task information and instructions allocated from the session node 20, and report a learning progress and resource status to the session node.

The worker node 30 may access the external network storage according to the allocated machine learning task information and instructions to download the machine learning task and training data files.

The learning progress status may include at least one of a Loss value, a learning accuracy, a validation value, and a progress rate.

5 is a block diagram of a worker node according to an embodiment. Referring to FIG. 5, the worker node 30 according to the embodiment may include a task executor 31 and a resource management module 32.

The task executor 31 may set an environment to perform machine learning according to the machine learning task information given from the session node 20. The task executor 31 may perform environment setting by checking a related Python version, checking a version of a machine learning framework, and downloading a training data set and a task file from an external network repository according to the machine learning task information.

The task executor 31 may perform learning execution, stop learning, and report on the progress of learning according to the machine learning execution command received from the session node 20. The learning execution of a task is a command that requests machine learning execution in the set task environment. Stop learning for a task is a command that requests stopping learning in progress. The task's training progress report is a command that requests the report of the log of the training execution result generated while performing machine learning. The task executor 31 manages a model file as a result of the progress of learning, and may update the corresponding task and a log of the training progress in the platform management database 15.

The resource management module 32 may periodically report the hardware and software resource status of the worker node 30 to the session node 20. The hardware resources are CPU/GPU computational processing capacity and real-time computational processing capacity utilization rate of the worker node 30, total memory and real-time memory utilization, and total disk and disk utilization. Real-time hardware resource status can be used as a key indicator for session nodes to allocate machine learning tasks to worker nodes. The softwareware resource status is a list of installed packages, libraries, and virtual environments when the worker node registers with the session node. When the software resource is newly installed or removed, the changed software resource status of the worker node may be transmitted to the session node and updated.

6 is a flowchart of a method for managing a machine learning platform according to an embodiment.

First, the worker node transmits resource information to the session node to register and request connection to the platform (S610).

Next, the session node checks whether the worker node is a node capable of performing machine learning through resource information (S620).

Next, the session node forms a group using at least one worker node capable of performing machine learning. In this case, the session node forms a group using at least one worker node using the same framework (S630).

Next, the master node registers a machine learning task and command through an external program (S640).

Next, the master node transmits the machine learning task information and instructions to the session node (S650).

Next, the session node analyzes the machine learning task information and instructions and allocates it to an executable worker node (S660).

Next, the worker node performs machine learning according to the machine learning task information and instructions allocated from the session node (S670).

Next, the worker node reports the learning progress and resource status to the session node (S680).

7 is a diagram illustrating a process of connecting a worker node according to an embodiment. Referring to FIG. 7, the worker node automatically requests a connection to the session node when it is executed. At this time, the worker node transmits its own hardware resource status and software resource status together with the connection request (S701).

The worker node manager of the session node receives a connection request from the worker node. The worker node manager of the session node checks whether a worker node has a framework that can execute machine learning tasks registered in the master node. When the worker node manager confirms that the worker node is capable of performing training, it adds the water node to the worker node list (S702).

At the same time, the resource monitoring manager of the master node updates information on the worker node managed by the session node to reflect the resource status of the new worker node (S703).

The resource monitoring manager provides a resource situation reflecting the connection of each session node and worker node to an external program (S704).

8 is a diagram for describing a process of assigning a machine learning task according to an embodiment. Referring to FIG. 8, when a user is registered with the platform through an external program, user information is transmitted to the client manager (S801).

User information is registered in the platform management database (S802).

The registered user may register a machine learning task through an external program (S803).

When the user registers the machine learning task, the data set and training file of the machine learning task are uploaded to the external network storage (S804).

In addition, when the machine learning task information including the location of the corresponding machine learning task file is transmitted to the task manager in the external network storage, the task manager registers the machine learning task information in the platform management database (S805).

The task manager checks the machine learning execution environment of the registered task and transmits the task information to the session node that manages the worker node capable of executing the machine learning (S806).

The worker node manager selects a worker node to allocate the task through the worker node list (S807).

The worker node manager delivers the task information to the selected worker node (S808).

The worker node accesses the external network storage based on the task information and downloads the task data set and training data file (S809).

9 is a diagram illustrating a process of executing a machine learning instruction according to an embodiment. Referring to Fig. 9, a remote execution command for a task registered in the master node is transmitted from an external program (S901).

The command processor requests task information from the platform management database, loads the task information, and transmits the task information and commands to the session node that received the task (S902 to 904).

The command analyzer analyzes the command and transmits the command to the worker node to which the task is assigned (S905).

Worker nodes execute registered tasks and received commands. Training execution result files such as a learning model output while performing a task are stored in a corresponding task folder in an external network storage (S906).

Logs such as intermediate result values that are output while performing learning are updated in the platform management database (S907).

The worker node transmits the command execution result to the command analyzer and reports whether the command has been correctly executed to the session node (S908).

The command analyzer transmits the command execution result to the command processor and transmits it to the master node (S909).

When the learning performance result is transmitted to an external program, the platform management database is accessed and the corresponding learning performance log is transmitted (S910 to 911).

The user can check the current machine learning progress and intermediate results through the learning performance log. In addition, the user may download a learning execution result file stored in an external network storage through an external program in real time (S912).

10 is a diagram illustrating an operation of a resource monitoring manager according to an embodiment of the present invention.

Referring to FIG. 10, the resource monitoring manager of the master node determines the necessity for load balancing of the machine learning task when the load of the machine learning task is concentrated on a specific framework based on the resource situation. The resource monitoring manager may determine the necessity of load balancing for the machine learning task according to at least one of the GFLOPS usage rate, the memory usage rate, and the disk usage rate (S1001).

The resource monitoring manager of the master node sends a command to add session node B to session nodes A and B for load balancing and issues a command to reconfigure worker nodes. This is to efficiently distribute the load according to the resource situation monitored in real time and the machine learning task assigned to the platform. The master node selects a worker node that can be assigned to session node B. The master node transmits the information of the session node and worker node to be changed for the reconfiguration command to each session node A and B (S1002).

The session node A transmits connection information of the newly allocated session node B to the worker node (S1003).

The worker node connects to the newly allocated session node B, and reports the resource status to the session node B (S1004).

The session node B adds a newly allocated worker node to the worker node list through the worker node manager and manages resources (S1005).

The term'~ unit' used in this embodiment refers to software or hardware components such as field-programmable gate array (FPGA) or ASIC, and'~ unit' performs certain roles. However,'~ part' is not limited to software or hardware. The'~ unit' may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays, and variables. The components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further divided into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

1: machine learning platform management system
2: external program
3: External network storage
10: master node
20: session node
30: worker node

Claims (10)

A master node for managing machine learning tasks and commands registered through an external program;
A session node that analyzes the machine learning task information and the command received from the master node and assigns it to an executable lower node; And
Machine learning platform management system comprising a worker node for performing machine learning according to the machine learning task information and the command allocated from the session node, and reporting a learning progress status and resource status to the session node.
Following paragraph 1,
The session node is a machine learning platform management system that analyzes the computational processing capacity according to the resource situation reported from the worker node, and allocates the machine learning task information and the instruction based on the analyzed computational processing capacity.
The method of claim 2,
The session node generates a list of worker nodes sequentially arranged according to the computational processing capability, and assigns the machine learning task information and the command according to the list of worker nodes.
The method according to claim 1 or 2,
The resource status is a machine including at least one of CPU resource information, GPU resource information, memory resource information, disk resource information, package version, library version, Python language version, framework type, framework version, operating system type, and operating system version. Learning platform management system.
The method of claim 1,
The session node is a machine learning platform management system that manages at least one worker node using the same framework.
The method of claim 1,
The worker node accesses an external network storage according to the allocated machine learning task information and the command to download the machine learning task and training data files.
The method of claim 1,
The learning progress status is a machine learning platform management system including at least one of a Loss value, a learning accuracy, a validation value, and a progress rate.
The method of claim 1,
The command is a machine learning platform management system including at least one of the allocation, execution, stopping of the machine learning task, and a learning progress report.
Requesting a connection by the worker node transmitting resource information to the session node;
Checking, by the session node, whether the worker node is a node capable of performing machine learning through the resource information;
Forming a group by the session node using at least one worker node capable of performing machine learning;
Receiving, by the master node, registering machine learning tasks and instructions through an external program;
Transmitting, by the master node, the machine learning task information and the command to the session node;
Analyzing, by the session node, the machine learning task information and the instruction and assigning it to an executable worker node;
Performing, by the worker node, machine learning according to the machine learning task information and the command allocated from the session node; And
And reporting, by the worker node, the learning progress status and the resource status to the session node.
The method of claim 9,
The step of forming the group,
A machine learning platform management method that forms a group using at least one worker node using the same framework.

Images