KR20200108789A - Method and computer program of processing program for single accelerator using dnn framework on plural accelerators - Google Patents

Abstract

The present disclosure relates to a method for processing a program for a single accelerator using a DNN framework in a plurality of accelerators. The method for processing a program for a single accelerator in a plurality of accelerators comprises the steps of: receiving a call for a deep learning operation function; receiving a call for an accelerator library function in order to execute the deep learning operation function in a single accelerator; assigning the accelerator library function to each of the plurality of accessible accelerators in response to the call for the accelerator library function; receiving, from each of the plurality of accelerators, intermediate result data of processing the accelerator library function; and generating result data for a called accelerator library function, on the basis of the received intermediate result data. According to the present invention, a process of developing a program is greatly shortened and thus, productivity of a program is increased.

Description

A method and computer program for processing a single accelerator program using the DNN framework in multiple accelerators {METHOD AND COMPUTER PROGRAM OF PROCESSING PROGRAM FOR SINGLE ACCELERATOR USING DNN FRAMEWORK ON PLURAL ACCELERATORS}

The present disclosure relates to a method and a computer program for processing a single accelerator program using a DNN (Deep Neural Network) framework in a plurality of accelerators, and specifically, an accelerator library function for executing a deep learning operation function in a single accelerator A method and a computer program for allocating an accelerator library function and data associated with the function to each of a plurality of accessible accelerators upon receiving a call to.

Recently, widely used DNN frameworks such as Tensorflow and Pytorch can provide operations based on cuDNN and cuBLAS, which are high-performance accelerator operation libraries running on a single accelerator, to users in the form of functions. Under such a conventional DNN framework, in order to utilize a plurality of accelerators, a user specifies each accelerator device installed in one computer, or specifies each computer in a network connected cluster to separately specify the work on each computer. Should be done. This requirement may arise because the library on which the DNN framework is based is the target of one accelerator, and thus task distribution must be performed at the DNN framework level or the user's application program level. In addition, depending on the configuration of the cluster system, the user must rewrite the program, and depending on the execution environment, code correction may be required, performance may deteriorate, or errors may occur.

In general, memory communication between accelerators and servers is handled internally by the DNN framework, so users do not need to be concerned. However, whether or not to share a deep neural network between multiple accelerators or each computer, designating tasks for each accelerator, and setting up cluster configurations are limited to providing functions related to them so that users (programmers) can directly process them at the application level. In other words, the DNN framework provides basic functions such as memory communication to use multiple accelerators mounted on a single computer or a cluster composed of multiple computers, but using the function to distribute tasks to multiple accelerators, etc. All additional work must be done by the user.

As the field of deep learning rapidly develops and changes, the network model or DNN framework to be implemented changes rapidly, and the execution environment such as the configuration or size of the cluster performing deep learning changes according to the development of technology. Therefore, it may be inevitable to modify and add code. Accordingly, the user must directly modify the code according to the execution environment or write the code to operate differently according to the options. This process takes a lot of time, and the structure of the code can become very complex, and maintenance and modification can be difficult accordingly.

The present disclosure provides a method for processing a single accelerator program using a DNN framework in a plurality of accelerators, and a computer program and system stored in a recording medium to solve the above problems.

It is an object of the present invention to create a deep learning learning/inference program for one accelerator by a programmer using a DNN framework, without modifying separate source codes (for example, both a user program and a DNN framework). , A method and a computer program for operating a program in a computer or cluster system in which a plurality of accelerators are installed are provided.

The present invention provides a method and computer program in which a high-performance accelerator operation library (eg, cuDNN, cuBLAS, etc.) for which the source code is not disclosed operates using a plurality of accelerators included in one computer or cluster system instead of a single accelerator. Is provided.

The present disclosure may be implemented in a variety of ways, including a method, apparatus, system, computer program, or computer readable storage medium storing instructions.

A method of processing a program for a single accelerator using a DNN framework according to an embodiment of the present disclosure in a plurality of accelerators includes the steps of receiving a call to a deep learning operation function, executing a deep learning operation function in a single accelerator. Receiving a call to an accelerator library function for, in response to a call to the accelerator library function, allocating an accelerator library function to each of a plurality of accessible accelerators, and processing the accelerator library function from each of the plurality of accelerators Receiving result data and generating result data for the called accelerator library function based on the received intermediate result data.

According to an embodiment, the allocating an accelerator library function to each of a plurality of accessible accelerators comprises: dividing the input data of the accelerator library function into a plurality of partial input data sets, and the accelerator library function and the divided plurality of parts. The step of assigning each of the input data sets to each of the plurality of accelerators, and receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators includes: the accelerator library function from each of the plurality of accelerators. And receiving intermediate result data of processing each of the plurality of partial input data sets by using.

According to an embodiment, the step of allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators comprises: analyzing a memory access pattern of the accelerator library function, and based on the analyzed access pattern. And allocating input data of the accelerator library function to each of the plurality of accessible accelerators before execution of the accelerator library function.

According to an embodiment, the step of allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators includes n partial input data sets allocated to each of the plurality of accelerators. In the case of including a set (where n is a natural number greater than or equal to 2), among n partial input data sets, the m-th partial input data set is processed by each of the plurality of accelerators, and the m+1-th partial input data set is plural. And assigning to each of the accelerators of (where m is a natural number less than n).

According to an embodiment, the allocating an accelerator library function to each of a plurality of accessible accelerators comprises: dividing the parameter data of the accelerator library function into a plurality of partial parameter data sets, and the accelerator library function and the divided plurality of parts. Assigning each of the parameter data sets to each of the plurality of accelerators.

According to an embodiment, receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators includes receiving intermediate result data of processing parameter data of the accelerator library function from each of the plurality of accelerators. And, generating result data for the called accelerator library function includes generating result data for intermediate result data obtained by processing parameter data of the accelerator library function.

According to an embodiment, generating result data for the called accelerator library function based on the received intermediate result data includes generating result data by concatenating the received intermediate result data.

According to an embodiment, generating result data for the called accelerator library function based on the received intermediate result data includes calculating the received intermediate result data to generate result data.

According to an embodiment, a plurality of accelerators are included in one computing device.

According to an embodiment, a plurality of accelerators are included in a cluster system including a plurality of computing devices.

A computer program stored in a computer-readable recording medium is provided in order to execute a method of processing the above-described single accelerator program in a plurality of accelerators in a computer according to an embodiment of the present disclosure.

According to some embodiments of the present disclosure, a DNN framework that is mainly used is commonly used, and a high-performance accelerator library in which the source code is not disclosed operates on a computer or a plurality of accelerators included in a cluster instead of a single accelerator. Can be.

According to some embodiments of the present disclosure, since a plurality of accelerators are automatically used at the library level, at the user application program level using the DNN framework, a program written for a single accelerator is appropriately distributed in an execution environment. Can be. In addition, since the operation is performed using a plurality of accelerators, execution time may be shortened. In addition, since the source code of the user application program does not need to be changed, the program development process is greatly shortened through this, thereby increasing the productivity of the program and making maintenance easier.

According to some embodiments of the present disclosure, since the operation of a library commonly used by the DNN framework is changed, it is not necessary to modify the source code of the DNN framework. In addition, a method of replacing or intercepting libraries previously linked during the compilation process of the DNN framework can be applied. This method can be easily applied to various types of DNN frameworks. Even when a new DNN framework is developed, processing of one accelerator and a plurality of accelerators can be unified.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals denote like elements, but are not limited thereto.
1 is a schematic diagram showing a configuration in which a host computing device is connected to communicate with a plurality of computing devices in order to provide a method for processing a program for a single accelerator in a plurality of accelerators according to an embodiment of the present disclosure.
2 is a block diagram illustrating an internal configuration of a computing device according to an embodiment of the present disclosure.
3 is a block diagram showing an internal configuration of a processor for providing a method of processing a single accelerator program using a DNN framework according to an embodiment of the present disclosure in a plurality of accelerators.
4 is a flowchart illustrating a method of processing a single accelerator program using a DNN framework according to an embodiment of the present disclosure in a plurality of accelerators.
5 is a diagram illustrating an example of dividing input data into a plurality of partial input data sets and allocating each of the divided plurality of partial input data sets to a plurality of accelerators according to an embodiment of the present disclosure.
6 is a diagram illustrating an example of allocating data when it is impossible to divide input data into a plurality of partial input data sets to a plurality of accelerators according to an embodiment of the present disclosure.
7 is a diagram illustrating an example of generating result data by connecting intermediate result data received from a plurality of accelerators according to an embodiment of the present disclosure.
8 is a diagram illustrating an example of generating result data by calculating intermediate result data received from a plurality of accelerators according to an embodiment of the present disclosure.
9 is a diagram illustrating an example of performing an operation in a plurality of accelerators and simultaneously copying a plurality of partial input data sets according to an embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, specific details for the implementation of the present disclosure will be described in detail. However, in the following description, when there is a possibility that the subject matter of the present disclosure may be unnecessarily obscure, detailed descriptions of widely known functions or configurations will be omitted.

In the accompanying drawings, the same or corresponding elements are assigned the same reference numerals. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, even if description of a component is omitted, it is not intended that such component is not included in any embodiment.

Advantages and features of the disclosed embodiments, and a method of achieving them will become apparent with reference to the embodiments described below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments make the present disclosure complete, and the present disclosure completely covers the scope of the invention to those skilled in the art. It is only provided to inform you.

The terms used in the present specification will be briefly described, and the disclosed embodiments will be described in detail. The terms used in the present specification have selected general terms that are currently widely used as possible while considering functions in the present disclosure, but this may vary according to the intention or precedent of a technician engaged in a related field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, not the name of a simple term.

In this specification, expressions in the singular include plural expressions, unless the context clearly specifies that they are singular. In addition, plural expressions include expressions in the singular unless clearly specified as plural in context. When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated.

In the present disclosure, a'cluster system' may include a plurality of computers connected through a network. Under such a client system, one client device can use the cluster system as a single computer. Since such a cluster system provides a connected user to use a plurality of computers as a single computer, a processing speed that is much improved than that of a single computer can be implemented.

In the present disclosure,'batch' means the number of samples before updating the network parameters of the model in deep learning. The training data set of a deep learning model can be divided into one or more batches. For example, in a deep learning model processing images, each batch size means the number of images processed at one time. If the batch size is 64, it may mean that 64 image files are sequentially placed in a continuous memory space. Accordingly, data divided by batch units is structurally the same, and data divided by batch units can be distributed to a plurality of accelerators without any special processing.

In the present disclosure, a'library wrapper' is a collection of subroutines or classes used to develop software, and the library wrapper may consist of a thin layer of code that converts the current interface of the library into a compatible interface. . For example, the library wrapper may include a function configured to change the behavior of a library commonly used by the DNN framework, so there is no need to modify the source code of the DNN framework itself. Accordingly, it is natural to those skilled in the art that the library wrapper of the present disclosure can be applied in a similar manner to various DNN frameworks.

1 illustrates a method for processing a single accelerator program in a plurality of accelerators according to an embodiment of the present disclosure, in which a host computing device 120 includes a plurality of computing devices 130_1, 130_2, ..., 130_n. It is a schematic diagram showing a configuration connected to enable communication with. Here, the plurality of accelerators may be included in the cluster system 100 including a plurality of computing devices 130_1, 130_2, ..., 130_n. For example, a plurality of accelerators may be included in at least some computing devices of the plurality of computing devices 130_1, 130_2, ..., 130_n. In addition, the cluster system 100 may include a network 110, a host computing device 120, and a plurality of computing devices 130_1, 130_2, ..., 130_n.

The network 110 may be configured to enable communication between the host computing device 120 included in the cluster system 100 and the plurality of computing devices 130_1, 130_2, ..., 130_n. Depending on the installation environment, for example, the network 110 is a wired network such as Ethernet, a wired home network (Power Line Communication), a telephone line communication device and RS-serial communication, a mobile communication network, a WLAN (Wireless LAN), It may consist of wireless networks such as Wi-Fi, Bluetooth and ZigBee, or a combination thereof. In other words, the communication method is not limited, and not only a communication method using a communication network (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network 110 may include, but also the host computing device 120 ) And the plurality of computing devices 130_1, 130_2, ..., 130_n may also include short-range wireless communication. For example, the network 110 is a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), It may include any one or more of the networks, such as the Internet. In addition, the network 110 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

The host computing device 120 may be configured to execute a program for a single accelerator using the DNN framework. In addition, the host computing device 120 is configured to communicate with a plurality of computing devices 130_1, 130_2, ..., 130_n, and included in the plurality of computing devices 130_1, 130_2, ..., 130_n). It may be configured to control the operation of a plurality of accelerators. For example, the host computing device 120 allocates an accelerator library function and input data to the plurality of computing devices 130_1, 130_2, ..., 130_n, and the plurality of computing devices 130_1, 130_2, ..., 130_n) Intermediate result data of processing the accelerator library function may be received from each, and result data for the called accelerator library function may be generated based on the received intermediate result data.

The plurality of computing devices 130_1, 130_2, ..., 130_n are computing devices that process information and communicate on the cluster system 100. The plurality of computing devices 130_1, 130_2, ..., 130_n may be configured in the form of a terminal such as a computer or a remote processing device. In addition, each of the plurality of computing devices 130_1, 130_2, ..., 130_n may independently perform information processing, etc., but other plurality of computing devices 130_1, 130_2, ..., 130_n through parallel programming You can also perform information processing, etc. while cooperating with them. Each of the plurality of computing devices 130_1, 130_2, ..., 130_n may execute communication for the operation of the deep learning application through the network 110. The plurality of computing devices 130_1, 130_2, ..., 130_n may correspond to any one of a transmission source, a destination, or a relay point of data.

2 is a block diagram illustrating an internal configuration of a computing device 200 according to an embodiment of the present disclosure. The plurality of accelerators 230_1, 230_2, ..., 230_n may be included in the computing device 200. According to an embodiment, the computing device 200 may refer to the host computing device 120. In another embodiment, the computing device 200 may refer to each of the plurality of computing devices 130_1, 130_2, ..., 130_n. As shown in FIG. 2, one computing device 200 may include a processor 210, a main memory 220, and a plurality of accelerators 230_1, 230_2, ..., 230_n.

The processor 210 may be configured as a general-purpose processor for arithmetic processing such as, for example, a CPU (Central Processing Unit, central processing unit), and the processor 210 includes a plurality of accelerators 230_1, 230_2, ... , 230_n) to control the operation of the plurality of accelerators 230_1, 230_2, ..., 230_n. Also, the main memory 220 may be connected to the processor 210. For example, the processor 210 may be connected to a plurality of accelerators 230_1, 230_2, ..., 230_n and/or the main memory 220 through a Peripheral Component Interconnect-Express (PCI-E) bus. Data for controlling the accelerators 230_1, 230_2, ..., 230_n and/or the main memory 220 may be transmitted and received.

According to an embodiment, the main memory 220 should be broadly interpreted to include any electronic component capable of storing electronic information. For example, main memory 220 includes random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read It may refer to various types of processor-readable media such as dedicated memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. If the processor 210 and/or the plurality of accelerators 230_1, 230_2, ..., 230_n can read information or write information to the memory, the main memory 220 is the processor 210 and/or the plurality of accelerators. It is said to be in electronic communication with (230_1, 230_2, ..., 230_n). In the present disclosure, the main memory 220 is a program executed by the processor 210 and/or a plurality of accelerators 230_1, 230_2, ..., 230_n (for example, a program for a single accelerator, a DNN framework, Any data and/or information (eg, program execution data, input data, parameter data, result data, etc.) associated with the accelerator library wrapper and accelerator library function may be stored.

Unlike a general-purpose CPU, the plurality of accelerators 230_1, 230_2, ..., 230_n may be configured as a processor specialized for a specific pattern of operation. For example, each of the plurality of accelerators 230_1, 230_2, ..., 230_n may include a GPU, FPGA, DSP, Intel Xeon Phi, TPU, NPU, multi-core CPU, etc. In addition, a plurality of accelerators 230_1, 230_2, ... Each of the accelerators 230_1, 230_2, ..., 230_n may have an accelerator memory (not shown) connected or included separately from the main memory.

3 is a block diagram showing an internal configuration of a processor 300 to provide a method for processing a single accelerator program 310 using a DNN framework 320 according to an embodiment of the present disclosure in a plurality of accelerators to be. According to an embodiment, the processor 300 may refer to the processor 210 of FIG. 2. As shown, the single accelerator program 310, the DNN framework 320, the accelerator library wrapper 330, and the accelerator library 340 may be operated or processed by the processor 300 or on the processor 300. have.

The single accelerator program 310 may mean a parallel programming model for accelerators 230 of various platforms in order to utilize the computational resources of the accelerator. According to an embodiment, the single accelerator program 310 may refer to a single GPU target user application. For example, the program for a single accelerator 310 may include any program that supports deep learning arithmetic functions for any accelerator, for example, OpenCL (Open Computing Language) and CUDA (Compute Unified Device Architecture). A parallel programming model or program such as) may be included, but is not limited thereto. Further, the deep learning operation function may be called through a single accelerator program 310.

The DNN framework 320 may include a set of software made to facilitate the creation and execution of deep learning applications. This DNN framework 320 applies a deep learning process or a deep learning computation function to an accelerator so that a developer can more easily use a parallel programming model or program that requires high proficiency, to provide a training process and inference. ) Process can be accelerated. For example, the DNN framework 320 may include DNN frameworks such as Caffe, Tensorflow, Pytorch, CNTK, and Theano, which are widely used recently, but is not limited thereto.

The accelerator library 340 may refer to a high-performance library that can be called through the DNN framework 320. The accelerator library 340 may provide a plurality of accelerator library functions to a user or developer. These accelerator library functions can be configured to run on a single accelerator.

According to an embodiment, the accelerator library 340 may refer to cuDNN and cuBLAS, but is not limited thereto. According to an embodiment, cuDNN may provide forward propagation and backward propagation such as a convolutional layer and a pooling layer in the form of an API. For example, cuDNN's cudnnConvolutionForward() function receives the weight corresponding to the layer from among the parameters of the network and the feature map (or image data) of the previous layer in batch units, and the next An output feature map to a layer can be output in batch units.

The accelerator library wrapper 330 may be configured to intercept the called accelerator library function and wrap the accelerator library function to enable parallel processing by a plurality of accelerators 230_1, 230_2, ..., 230_n. . Under this structure, the code written for the single accelerator 230 at the level of the single accelerator program 310 and the DNN framework 320 will be executed in a plurality of accelerators 230_1, 230_2, ..., 230_n at the library level. I can. For example, the accelerator library wrapper 330 may refer to a GPU library wrapper. In one embodiment, when a single accelerator program 310 is processed using the accelerator library wrapper 330 using the accelerator library wrapper 330, the accelerator library is modified by modifying the build process of the DNN framework 320. A single accelerator program 310 can be processed in parallel in a plurality of accelerators 230_1, 230_2, ..., 230_n, without modification of functions and source codes (eg, user programs and DNN framework 320). have. For example, the processor 210 may allocate an accelerator library function to each of the plurality of accelerators 230_1, 230_2, ..., 230_n.

4 is a flowchart illustrating a method 400 of processing a program for a single accelerator using a DNN framework in a plurality of accelerators according to an embodiment of the present disclosure. This method 400 may be performed by a processor. As shown, method 400 may begin with receiving a call to a deep learning computation function (S410). For example, a DNN framework running on a processor may receive a call to a deep learning operation function from a single accelerator program.

Then, in step S420, a call to an accelerator library function for executing a deep learning operation function in a single accelerator may be received. For example, an accelerator library wrapper running on a processor may receive a call from the DNN framework to an accelerator library function for executing a deep learning operation function in a single accelerator.

Next, in step S430, in response to a call of the accelerator library function, an accelerator library function may be allocated to each of a plurality of accessible accelerators. In one embodiment, the accelerator library wrapper on the processor receives a call to the accelerator library function for executing the deep learning computation function in a single accelerator from the DNN framework, and assigns the accelerator library function to each of the plurality of accelerators. have. At this time, the accelerator library wrapper may divide the input data of the accelerator library function into a plurality of partial input data sets, and allocate each of the divided partial input data sets to each of the plurality of accelerators. For example, if the input data of an accelerator library function is received in batch units, the accelerator library wrapper can distribute the input data to each of the plurality of accelerators in batch units, divide it into one or more data sets, and allocate them. have.

According to an embodiment, the accelerator library wrapper may allocate the accelerator library function and each of the divided plurality of partial input data sets to each of the plurality of accelerators. Here, in allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators, the memory access pattern of the accelerator library function is analyzed and based on the analyzed access pattern, before execution of the accelerator library function Input data of an accelerator library function may be assigned to each of a plurality of accessible accelerators. For example, the access pattern of the input and output of the DNN-related accelerator library function to memory uses a compiler analysis technique and/or a profiling technique (e.g., a technique that detects the input/output pattern of memory by executing a function in advance). Can be analyzed. As another example, before execution of the accelerator library function, data may be pre-distributed to a plurality of accelerators using an access pattern analyzed by a compiler analysis and/or profiling technique. Due to the behavioral characteristics of the DNN framework, tasks of the same pattern may be repeated for several iterations during the entire learning or inference process. When a memory access pattern for the first or several iterations (for example, 3 times) is identified using these characteristics, the memory access pattern of each accelerator library function can be analyzed during the entire learning or inference process.

Next, in step S440, intermediate result data of processing the accelerator library function may be received from each of the plurality of accelerators. According to an embodiment, the accelerator library wrapper may receive intermediate result data from a plurality of accelerators that have processed the accelerator library function. For example, the accelerator library wrapper may receive intermediate result data obtained by processing each of a plurality of partial input data sets using an accelerator library function in each of the plurality of accelerators. As another example, when output data of an accelerator library function is output in batch units, the accelerator library wrapper may receive intermediate result data obtained by processing each of the input data sets from a plurality of accelerators in batch units.

Finally, in step S450, result data for the called accelerator library function may be generated based on the received intermediate result data. For example, the accelerator library wrapper may generate result data based on intermediate result data received from a plurality of accelerators, which will be described later in more detail with reference to FIGS. 7 and 8.

A plurality of accelerators 530_1, 530_2, ..., 530_n; where n is a natural number greater than or equal to 2 shown in FIGS. 5 to 8 to be described later are included in one computing device or a cluster including a plurality of computing devices Can be included in the system. Here, the plurality of accelerators 530_1, 530_2, ..., 530_n may include an accelerator accessible by a computing device (node) on which the DNN framework is operated.

5 is a diagram illustrating an input data 510 according to an embodiment of the present disclosure divided into a plurality of partial input data sets 520_1, 520_2, ..., 520_n, and a plurality of divided partial input data sets 520_1, 520_2. , ..., 520_n) is a diagram showing an example of allocating each of the plurality of accelerators (530_1, 530_2, ..., 530_n).

According to an embodiment, as shown, the input data 510 of the accelerator library function may be divided into a plurality of partial input data sets 520_1, 520_2, ..., 520_n. Each of the divided plurality of partial input data sets 520_1, 520_2, ..., 520_n and a corresponding accelerator library function may be allocated to each of the plurality of accelerators 530_1, 530_2, ..., 530_n.

According to an embodiment, when the input data 510 is received in batch units, the accelerator library wrapper may evenly distribute the input data 510 to a plurality of accelerators 530_1, 530_2, ..., 530_n. For example, in Tensorflow or Pytorch, the memory layout of the data corresponding to the neuron is generally "NCHW" or "NHWC". Since "N" is in front of it, it may correspond to a layout in which the highest dimension is the layout. . Accordingly, assuming that there are two accelerators included in the cluster system, the entire data can be divided into two consecutive parts and distributed to two accelerators.

In another embodiment, when the processing speed of the partial input data to be distributed is different, or there is a difference in processing performance due to a network configuration of a cluster system, etc., a plurality of partial input data sets 520_1, 520_2, .. ., 520_n) may be differentially distributed to a plurality of accelerators.

6 illustrates an example of allocating data when the input data 610 cannot be divided into a plurality of partial input data sets to a plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure. It is a drawing showing.

In an embodiment, when the input data 610 cannot be divided into batch units, the accelerator library wrapper may allocate the same input data 610 to the plurality of accelerators 530_1, 530_2, ..., 530_n. . Here, when the input data 610 cannot be divided into batch units, the corresponding input data 610 may be required by the plurality of accelerators 530_1, 530_2, ..., 530_n, and the network parameter is mainly used in the DNN. This may be the case. For example, in Tensorflow or Pytorch, the memory layout of parameter data is generally "CKHW" or "HWKC", which is independent of batch and cannot be divided into batch units. Therefore, a copy of the input data 610 can be allocated to each of the plurality of accelerators 530_1, 530_2, ..., 530_n so that the plurality of accelerators 530_1, 530_2, ..., 530_n use the same data. have.

According to another embodiment, the accelerator library wrapper divides the parameter data (ie, weights; not shown) of the accelerator library function into a plurality of partial parameter data sets, and divides each of the divided partial parameter data sets into a plurality of accelerators 530_1, 530_2, ..., 530_n). In this case, the accelerator library wrapper may provide an accelerator library function associated with the divided parameter data set to the plurality of accelerators 530_1, 530_2, ..., 530_n. In addition, the accelerator library wrapper provides the input data 610 of the accelerator library function associated with the divided parameter data set together with the parameter data set and the accelerator library function divided into a plurality of accelerators (530_1, 530_2, ..., 530_n). can do. Here, as shown, the input data 610 may not be divided. In contrast, the accelerator library wrapper divides the input data 610 of the accelerator library function associated with the divided parameter data set into a plurality of partial input data sets, and divides the divided plurality of partial input data sets, the divided parameter data set and It may be provided to a plurality of accelerators 530_1, 530_2, ..., 530_n together with an accelerator library function.

7 is a connection between intermediate result data 710_1, 710_2, ..., 710_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure, and result data 720 ) Is a diagram showing an example of generating. According to an embodiment, the accelerator library wrapper receives intermediate result data 710_1, 710_2, ..., 710_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n, and received intermediate result data By concatenating (710_1, 710_2, ..., 710_n), result data 720 may be generated. For example, in the case of a convolution function among accelerator library functions provided by CuDNN, intermediate result data (710_1, 710_2, ..., 710_n) received from a plurality of accelerators (530_1, 530_2, ..., 530_n) are connected. By doing so, result data 720 may be generated.

In one embodiment, the accelerator library wrapper 330, when the intermediate result data (710_1, 710_2, ..., 710_n) is parallelized in batch units, the intermediate result data (710_1, 710_2, ..., 710_n) By concatenating, result data 720 may be generated. For example, when intermediate result data (710_1, 710_2, ..., 710_n) are parallelized in batch units, intermediate result data (710_1, 710_2, ..., 710_n) are simply connected as result data (710). Can be saved.

8 illustrates intermediate result data 810_1, 810_2, ..., 810_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure, and result data 820 ) Is a diagram showing an example of generating. As shown in FIG. 8, the accelerator library wrapper processes the parameter data of the accelerator library function from each of the plurality of accelerators 530_1, 530_2, ..., 530_n, and intermediate result data 810_1, 810_2, ..., 810_n) may be received, and intermediate result data 810_1, 810_2, ..., 810_n obtained by processing the parameter data of the accelerator library function may be calculated, and result data 820 may be generated. This calculation process may be referred to as reduction. For example, in the case of max pooling among accelerator library functions provided by CuDNN, intermediate result data (810_1, 810_2, ..., 810_n) received from multiple accelerators (530_1, 530_2, ..., 530_n) are calculated. By processing, result data 820 may be generated.

According to an embodiment, when intermediate result data 810_1, 810_2, ..., 810_n are not parallelized in the arrangement direction, an operation for generating gradient data of a network parameter required for a learning process may be used. A plurality of accelerators (530_1, 530_2, ..., 530_n) calculates the network parameter gradient from the input data distributed to them, and the intermediate result data (810_1, 810_2, ..., 810_n) can be exchanged and summed. have.

9 is an example of performing an operation in a plurality of accelerators 930_1, ..., 930_n; where n is a natural number equal to or greater than 2 according to an embodiment of the present disclosure and simultaneously copying a plurality of partial input data sets It is a figure showing. According to an embodiment, when the partial input data set 910 allocated to each of the plurality of accelerators 930_1, ..., 930_n includes n partial input data sets (where n is a natural number of 2 or more) ), of the n partial input data sets 910, the m-th partial input data set 920_1 is processed by each of the plurality of accelerators 930_1, ..., 930_n, and the m+1th partial input data set ( Allocating 920_2) to each of the plurality of accelerators 930_1, ..., 930_n (here, m is a natural number less than n) may be included. To this end, before allocating each of the plurality of partial input data sets 910 to each of the plurality of accelerators 930_1, ..., 930_n, a memory access pattern of an accelerator library function to be processed may be analyzed.

In one embodiment, the accelerator library wrapper 330 allocates the m-th partial input data set 920_1, which is a part of the plurality of partial input data sets 910, to the accelerator 930_1 based on the analyzed memory access pattern. You can handle it. While the m-th partial input data set 920_1 is processed by the accelerator 930_1, the accelerator library wrapper 330 may allocate the m+1th partial input data set 920_2 to the accelerator 930_1. For example, due to the mechanical properties of the plurality of accelerators (930_1, ..., 930_n), the copy operation and the operation operation of the memory in the accelerator are processed separately, so that the plurality of accelerators (930_1, .. ., 930_n), the total execution time of the deep learning application can be reduced by simultaneously processing the copy of the memory while the operation is being performed. Under this operation method, as shown, a plurality of partial input data sets 920_1 and 920_3 are allocated to each of a plurality of accelerators 930_1 and 930_n, and a plurality of allocated partial input data sets 920_1 and 920_3 are During processing, another plurality of partial input data sets 920_2 and 920_4 may be allocated to each of the plurality of accelerators 930_1 and 930_n.

A method of processing a single accelerator program using the above-described DNN framework in a plurality of accelerators may be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the technical field to which the present invention belongs.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those of skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure , Computer, or a combination thereof.

Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may include a general purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or It may be implemented or performed in any combination of those designed to perform the functions described in. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in connection with the DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer-readable medium such as programmable read-only memory, erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, etc. It can also be implemented as stored instructions. The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

When implemented in software, the techniques may be stored on a computer-readable medium as one or more instructions or code or transmitted through a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer-readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. It may include any other medium that may be used for transfer or storage to and accessible by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared, wireless, and microwave, coaxial cable , Fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, wireless, and microwave are included within the definition of the medium. Disks and discs as used herein include CDs, laser disks, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray disks, where disks are usually magnetic It reproduces data optically, whereas discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other known type of storage medium. An exemplary storage medium may be coupled to a processor such that the processor can read information from or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may also reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and storage medium may exist as separate components in the user terminal.

Although the above-described embodiments have been described as utilizing aspects of the currently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto, and may be implemented in connection with any computing environment such as a network or a distributed computing environment. . Furthermore, aspects of the subject matter in this disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and portable devices.

In the present specification, the present disclosure has been described in connection with some embodiments, but various modifications and changes may be made without departing from the scope of the present disclosure as understood by those of ordinary skill in the art to which the present disclosure belongs. In addition, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

100: cluster system
110: network
120: host computing device
130, 130_1, 130_2, ..., 130_n: computing device
200: computing device
210: processor
220: main memory
230, 230_1, 230_2, ..., 230_n: accelerator
310: Program for a single accelerator
320: DNN framework
330: accelerator library wrapper
340: accelerator library

Claims (11)

In a method for processing a single accelerator program using a DNN framework in a plurality of accelerators,
Receiving a call to a deep learning operation function;
Receiving a call to an accelerator library function for executing the deep learning operation function in a single accelerator;
Allocating the accelerator library function to each of a plurality of accessible accelerators in response to a call of the accelerator library function;
Receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators; And
Generating result data for the called accelerator library function based on the received intermediate result data,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
Allocating the accelerator library function to each of the plurality of accessible accelerators,
Dividing the input data of the accelerator library function into a plurality of partial input data sets; And
Assigning each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators
Including,
The step of receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators may include processing each of the plurality of partial input data sets by using the accelerator library function in each of the plurality of accelerators. Including the step of receiving result data,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 2,
Allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators,
Analyzing a memory access pattern of the accelerator library function; And
Allocating input data of the accelerator library function to each of the plurality of accessible accelerators before execution of the accelerator library function based on the analyzed access pattern,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 2,
Allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators,
When the partial input data set allocated to each of the plurality of accelerators includes n partial input data sets (where n is a natural number of 2 or more), from the n partial input data sets, an m-th partial input data set Processing in each of the plurality of accelerators and simultaneously allocating an m+1th partial input data set to each of the plurality of accelerators (where m is a natural number less than n),
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
Allocating the accelerator library function to each of the plurality of accessible accelerators,
Dividing the parameter data of the accelerator library function into a plurality of partial parameter data sets; And
Allocating each of the accelerator library function and the divided plurality of partial parameter data sets to each of the plurality of accelerators,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 5,
The step of receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators includes receiving intermediate result data of processing parameter data of the accelerator library function from each of the plurality of accelerators,
Generating result data for the called accelerator library function comprises generating result data for intermediate result data obtained by processing parameter data of the accelerator library function,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
Based on the received intermediate result data, generating result data for the called accelerator library function comprises generating the result data by concatenating the received intermediate result data,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
On the basis of the received intermediate result data, generating result data for the called accelerator library function includes calculating the received intermediate result data to generate the result data,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
The plurality of accelerators are included in one computing device,
A method of processing a program for a single accelerator in multiple accelerators.
The method of claim 1,
The plurality of accelerators are included in a cluster system including a plurality of computing devices,
A method of processing a program for a single accelerator in multiple accelerators.
A computer program stored in a computer-readable recording medium to execute a method of processing a single accelerator program using a DNN framework according to any one of claims 1 to 10 in a plurality of accelerators.

Images