KR102376527B1 - Method and computer program of processing program for single accelerator using dnn framework on plural accelerators - Google Patents

Abstract

The present disclosure relates to a method of processing a program for a single accelerator using a DNN framework in a plurality of accelerators. A method for processing a program for a single accelerator in a plurality of accelerators includes receiving a call to a deep learning computation function, receiving a call to an accelerator library function for executing a deep learning computation function in a single accelerator, the accelerator responsive to the call of the library function, assigning an accelerator library function to each of a plurality of accessible accelerators, receiving intermediate result data of processing an accelerator library function from each of the plurality of accelerators, and receiving the intermediate result data based on it, generating result data for the called accelerator library function.

Description

Method and computer program for processing a program for a single accelerator using the DNN framework in multiple accelerators

The present disclosure relates to a method and a computer program for processing a program for a single accelerator using a deep neural network (DNN) framework in a plurality of accelerators, and specifically, an accelerator library function for executing a deep learning operation function in a single accelerator A method and computer program for allocating accelerator library functions and data associated with these functions to each of a plurality of accessible accelerators upon receiving a call to.

Tensorflow and Pytorch, which are recently widely used DNN frameworks, can provide users with operations in the form of functions based on cuDNN and cuBLAS, which are high-performance accelerator operation libraries that run on a single accelerator. Under such a conventional DNN framework, in order to utilize a plurality of accelerators, a user specifies an accelerator device installed in one computer, or specifies each computer in a network-connected cluster to separately specify an operation in each computer should do This requirement can occur because the library on which the DNN framework is based is an accelerator target, so task distribution must be performed at the DNN framework level or the user's application level. In addition, depending on the configuration of the cluster system, the user has to rewrite the program, and depending on the execution environment, code modification is required, performance is deteriorated, or an error may occur.

In general, memory communication between accelerators and servers is handled internally by the DNN framework, so users do not need to worry about it. However, whether a deep neural network is shared between multiple accelerators or each computer, task assignments for each accelerator, and cluster configuration settings are only provided so that users (programmers) can directly process at the application level. In other words, basic functions such as memory communication are provided to use multiple accelerators mounted on a single computer or a cluster composed of multiple computers internally within the DNN framework. Any additional work must be done by the user.

As the field of deep learning rapidly develops and changes, the network model or DNN framework that is the target of implementation changes rapidly, and the execution environment such as the configuration or size of the cluster performing deep learning changes according to the development of technology Therefore, code modifications and additions may be unavoidable. Accordingly, the user must either directly modify the code according to the execution environment or write the code to operate differently depending on the options. This process takes a lot of time, and the structure of the code can become very complex, which can make maintenance and change difficult.

The present disclosure provides a method for processing a single accelerator program using a DNN framework for solving the above problems in a plurality of accelerators, a computer program stored in a recording medium, and a system.

An object of the present invention is that when a programmer writes a deep learning learning/inference program for one accelerator by using the DNN framework, it is possible without modification of a separate source code (for example, both the user program and the DNN framework). , a method and computer program for allowing the program to operate in a computer or cluster system in which a plurality of accelerators are installed are provided.

The present invention provides a method and computer program for enabling a high-performance accelerator operation library (eg, cuDNN, cuBLAS, etc.) whose source code is not disclosed to operate using a plurality of accelerators included in one computer or cluster system instead of a single accelerator. provided

The present disclosure may be embodied in a variety of ways, including a method, apparatus, system, computer program, or computer-readable storage medium storing instructions.

A method of processing a program for a single accelerator using a DNN framework according to an embodiment of the present disclosure in a plurality of accelerators, receiving a call to a deep learning operation function, executing a deep learning operation function in a single accelerator receiving a call to an accelerator library function for; in response to the call of the accelerator library function, assigning an accelerator library function to each of a plurality of accessible accelerators; intermediate processing of an accelerator library function from each of the plurality of accelerators; receiving result data and generating, based on the received intermediate result data, result data for the called accelerator library function.

According to one embodiment, assigning an accelerator library function to each of a plurality of accessible accelerators comprises partitioning input data of the accelerator library function into a plurality of partial input data sets and the accelerator library function and the partitioned plurality of portions. and assigning each of the input data sets to each of a plurality of accelerators, wherein receiving intermediate result data of processing an accelerator library function from each of the plurality of accelerators comprises: an accelerator library function in each of the plurality of accelerators; and receiving intermediate result data obtained by processing each of the plurality of partial input data sets using the method.

According to one embodiment, the assigning each of the accelerator library function and the partitioned plurality of partial input data sets to each of the plurality of accelerators comprises: analyzing a memory access pattern of the accelerator library function and based on the analyzed access pattern and allocating input data of the accelerator library function to each of a plurality of accessible accelerators prior to execution of the accelerator library function.

According to an embodiment, the step of allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators includes: The partial input data set allocated to each of the plurality of accelerators includes n partial input data In the case of including a set (where n is a natural number greater than or equal to 2), among the n partial input data sets, the mth partial input data set is processed by each of the plurality of accelerators, and the m+1th partial input data set is plural. assigning to each of the accelerators of , where m is a natural number less than n.

According to one embodiment, assigning an accelerator library function to each of a plurality of accessible accelerators comprises: partitioning parameter data of the accelerator library function into a plurality of partial parameter data sets; and the accelerator library function and the partitioned plurality of parts. assigning each of the parameter data sets to each of the plurality of accelerators.

According to an embodiment, receiving intermediate result data of processing an accelerator library function from each of the plurality of accelerators comprises receiving intermediate result data of processing parameter data of the accelerator library function from each of the plurality of accelerators and generating result data for the called accelerator library function includes generating result data for intermediate result data obtained by processing parameter data of the accelerator library function.

According to one embodiment, based on the received intermediate result data, generating the result data for the called accelerator library function comprises concatenating the received intermediate result data to generate the result data.

According to one embodiment, based on the received intermediate result data, generating result data for the called accelerator library function includes calculating the received intermediate result data to generate result data.

According to an embodiment, a plurality of accelerators are included in one computing device.

According to an embodiment, the plurality of accelerators are included in a cluster system including a plurality of computing devices.

A computer program stored in a computer-readable recording medium is provided in order to execute a method of processing the above-described single accelerator program in a plurality of accelerators in a computer according to an embodiment of the present disclosure.

According to some embodiments of the present disclosure, a mainly used DNN framework is commonly used, and a high-performance accelerator library whose source code is not disclosed operates on a plurality of accelerators included in one computer or cluster instead of one accelerator. can be

According to some embodiments of the present disclosure, since a plurality of accelerators are automatically utilized at the library level, at the user application level using the DNN framework, one program written for a single accelerator is appropriately distributed processing in the execution environment can be In addition, since the calculation is performed using a plurality of accelerators, the execution time may be shortened. In addition, since the source code of the user application does not need to be changed, the program development process can be greatly shortened through this, thereby increasing the productivity of the program and making maintenance easier.

According to some embodiments of the present disclosure, since the operation of the library commonly used by the DNN framework is changed, the source code of the DNN framework does not need to be modified. In addition, a method of replacing or intercepting a library previously linked in the process of compiling the DNN framework may be applied. This method can be easily applied to various kinds of DNN frameworks. Even in the case of developing a new DNN framework, processing of one accelerator and a plurality of accelerators may be unified.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals denote like elements, but are not limited thereto.
1 is a schematic diagram illustrating a configuration in which a host computing device is connected to communicate with a plurality of computing devices in order to provide a method of processing a program for a single accelerator in a plurality of accelerators according to an embodiment of the present disclosure.
2 is a block diagram illustrating an internal configuration of a computing device according to an embodiment of the present disclosure.
3 is a block diagram illustrating an internal configuration of a processor for providing a method of processing a program for a single accelerator using a DNN framework in a plurality of accelerators according to an embodiment of the present disclosure.
4 is a flowchart illustrating a method of processing a program for a single accelerator using a DNN framework in a plurality of accelerators according to an embodiment of the present disclosure.
5 is a diagram illustrating an example of dividing input data into a plurality of partial input data sets and allocating each of the divided plurality of partial input data sets to a plurality of accelerators according to an embodiment of the present disclosure;
6 is a diagram illustrating an example of allocating data to a plurality of accelerators when it is not possible to divide input data into a plurality of partial input data sets according to an embodiment of the present disclosure;
7 is a diagram illustrating an example of generating result data by connecting intermediate result data received from a plurality of accelerators according to an embodiment of the present disclosure.
8 is a diagram illustrating an example of generating result data by calculating intermediate result data received from a plurality of accelerators according to an embodiment of the present disclosure.
9 is a diagram illustrating an example of performing an operation in a plurality of accelerators and copying a plurality of partial input data sets at the same time according to an embodiment of the present disclosure;

Hereinafter, specific contents for carrying out the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, the same or corresponding components are assigned the same reference numerals. In addition, in the description of the embodiments below, overlapping description of the same or corresponding components may be omitted. However, even if descriptions regarding components are omitted, it is not intended that such components are not included in any embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, and only the present embodiments allow the present disclosure to be complete, and the present disclosure will provide those of ordinary skill in the art to fully understand the scope of the invention. It is only provided to inform you.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail. Terms used in this specification have been selected as currently widely used general terms as possible while considering the functions in the present disclosure, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

References in the singular herein include plural expressions unless the context clearly dictates the singular. Also, the plural expression includes the singular expression unless the context clearly dictates the plural. In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In the present disclosure, a 'cluster system' may include a plurality of computers connected through a network. Under such a client system, one client device can use the cluster system as one computer. Since such a cluster system provides connected users to use a plurality of computers as if they were one computer, much improved processing speed than that of one computer can be implemented.

In the present disclosure, 'batch' means the number of samples before updating the network parameters of the model in deep learning. The training data set of a deep learning model can be divided into one or more batches. For example, in a deep learning model that processes images, each batch size means the number of images processed at one time. If the batch size is 64, it may mean that 64 image files are sequentially placed in a contiguous memory space. Accordingly, data divided in batch units is structurally the same, and data divided in batch units can be distributed to a plurality of accelerators without special processing.

In this disclosure, a 'library wrapper' is a collection of subroutines or classes used to develop software, and the library wrapper can be composed of a thin layer of code that converts the current interface of a library into a compatible interface. . For example, the library wrapper may contain functions configured to change the behavior of libraries commonly used by the DNN framework, so there is no need to modify the source code of the DNN framework itself. Accordingly, it is natural for those skilled in the art that the library wrapper of the present disclosure can be applied to various DNN frameworks in a similar manner.

1 is a diagram illustrating a host computing device 120 in a plurality of computing devices 130_1, 130_2, ..., 130_n in order to provide a method for processing a program for a single accelerator in a plurality of accelerators according to an embodiment of the present disclosure. It is a schematic diagram showing the configuration connected so as to be able to communicate with the Here, the plurality of accelerators may be included in the cluster system 100 including the plurality of computing devices 130_1 , 130_2 , ..., 130_n. For example, the plurality of accelerators may be included in at least some of the plurality of computing devices 130_1 , 130_2 , ..., 130_n. Also, the cluster system 100 may include a network 110 , a host computing device 120 , and a plurality of computing devices 130_1 , 130_2 , ..., 130_n .

The network 110 may be configured to enable communication between the host computing device 120 included in the cluster system 100 and the plurality of computing devices 130_1 , 130_2 , ..., 130_n. Network 110, depending on the installation environment, for example, Ethernet (Ethernet), wired home network (Power Line Communication), telephone line communication device and wired networks such as RS-serial communication, mobile communication network, WLAN (Wireless LAN), It may consist of a wireless network such as Wi-Fi, Bluetooth and ZigBee, or a combination thereof. In other words, the communication method is not limited, and the host computing device 120 as well as a communication method using a communication network (eg, mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network 110 may include. ) and short-range wireless communication between the plurality of computing devices 130_1, 130_2, ..., 130_n may also be included. For example, the network 110 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), may include any one or more of networks such as the Internet. In addition, the network 110 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. not limited

The host computing device 120 may be configured to execute a program for a single accelerator using the DNN framework. In addition, the host computing device 120 is configured to be able to communicate with the plurality of computing devices 130_1, 130_2, ..., 130_n, and included in the plurality of computing devices 130_1, 130_2, ..., 130_n). It may be configured to control operation of a plurality of accelerators. For example, the host computing device 120 allocates an accelerator library function and input data to the plurality of computing devices 130_1, 130_2, ..., 130_n, and the plurality of computing devices 130_1, 130_2, ..., 130_n) may receive intermediate result data of processing the accelerator library function from each, and generate result data for the called accelerator library function based on the received intermediate result data.

The plurality of computing devices 130_1 , 130_2 , ..., 130_n are computing devices that perform information processing and communication on the cluster system 100 . The plurality of computing devices 130_1, 130_2, ..., 130_n may be configured in the form of a terminal such as a computer or a remote processing device. In addition, each of the plurality of computing devices 130_1, 130_2, ..., 130_n may independently perform information processing, etc., but through parallel programming, a plurality of other computing devices 130_1, 130_2, ..., 130_n) Information processing, etc. can also be carried out in cooperation with others. Each of the plurality of computing devices 130_1 , 130_2 , ..., 130_n may execute communication for the operation of the deep learning application through the network 110 . The plurality of computing devices 130_1 , 130_2 , ..., 130_n may correspond to any one of a data transmission source, a destination, or a relay point.

2 is a block diagram illustrating an internal configuration of a computing device 200 according to an embodiment of the present disclosure. The plurality of accelerators 230_1 , 230_2 , ..., 230_n may be included in the computing device 200 . According to an embodiment, the computing device 200 may refer to the host computing device 120 . In another embodiment, the computing device 200 may refer to each of the plurality of computing devices 130_1 , 130_2 , ..., 130_n. As shown in FIG. 2 , one computing device 200 may include a processor 210 , a main memory 220 , and a plurality of accelerators 230_1 , 230_2 , ..., 230_n .

The processor 210 may be configured as a general-purpose processor for arithmetic processing such as, for example, a central processing unit (CPU), and the processor 210 includes a plurality of accelerators 230_1, 230_2, ... , 230_n) to control the operation of the plurality of accelerators 230_1, 230_2, ..., 230_n. In addition, the processor 210 may be connected to the main memory 220 . For example, the processor 210 may be connected to a plurality of accelerators 230_1 , 230_2 , ..., 230_n and/or the main memory 220 through a Peripheral component interconnect-Express (PCI-E) bus, and may be connected to a plurality of Data for controlling the accelerators 230_1 , 230_2 , ..., 230_n and/or the main memory 220 may be transmitted/received.

According to one embodiment, main memory 220 should be broadly interpreted to include any electronic component capable of storing electronic information. For example, main memory 220 may include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read It may refer to various types of processor-readable media, such as dedicated memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. If information can be read from or written to the processor 210 and/or the plurality of accelerators 230_1, 230_2, ..., 230_n, the main memory 220 is the processor 210 and/or the plurality of accelerators. It is said to be in electronic communication with (230_1, 230_2, ..., 230_n). In the present disclosure, the main memory 220 includes a program (eg, a program for a single accelerator, a DNN framework, It may store any data and/or information (eg, program execution data, input data, parameter data, result data, etc.) associated with accelerator library wrappers and accelerator library functions, etc.).

The plurality of accelerators 230_1 , 230_2 , ..., 230_n may be configured as a processor specialized for a specific pattern operation, unlike a general-purpose CPU. For example, each of the plurality of accelerators (230_1, 230_2, ..., 230_n) may include a GPU, FPGA, DSP, Intel Xeon Phi, TPU, NPU, multi-core CPU, etc. In addition, a plurality of Each of the accelerators 230_1 , 230_2 , ..., 230_n may be connected to or included in an accelerator memory (not shown) separately from the main memory.

3 is a block diagram showing the internal configuration of the processor 300 to provide a method for processing a program 310 for a single accelerator using the DNN framework 320 according to an embodiment of the present disclosure in a plurality of accelerators am. According to an embodiment, the processor 300 may refer to the processor 210 of FIG. 2 . As shown, the program 310 for a single accelerator, the DNN framework 320 , the accelerator library wrapper 330 and the accelerator library 340 may be operated or processed by or on the processor 300 . there is.

The program 310 for a single accelerator may refer to a parallel programming model for the accelerator 230 of various platforms in order to utilize the computation resources of the accelerator. According to an embodiment, the single accelerator program 310 may refer to a single GPU target user application program. For example, the program 310 for a single accelerator may include any program supporting a deep learning computation function for any accelerator, for example, Open Computing Language (OpenCL) and Compute Unified Device Architecture (CUDA). ) may include a parallel programming model or program, such as, but not limited to. In addition, the deep learning operation function may be called through the program 310 for a single accelerator.

The DNN framework 320 may include a collection of built-in software that facilitates the creation and execution of deep learning applications. The DNN framework 320 applies a deep learning processing or deep learning operation function to an accelerator so that a developer can more easily use a parallel programming model or program that requires high proficiency in a training process and inference. ) can accelerate the process. For example, the DNN framework 320 may include, but is not limited to, DNN frameworks such as Caffe, Tensorflow, Pytorch, CNTK, and Theano that are widely used recently.

The accelerator library 340 may refer to a high-performance library that may be called through the DNN framework 320 . The accelerator library 340 may provide a plurality of accelerator library functions to a user or a developer. These accelerator library functions can be configured to run on a single accelerator.

According to an embodiment, the accelerator library 340 may refer to cuDNN and cuBLAS, but is not limited thereto. According to an embodiment, cuDNN may provide forward propagation and backward propagation of a convolutional layer and a pooling layer in the form of an API. For example, the cudnnConvolutionForward() function of cuDNN receives the weight corresponding to the corresponding layer among the parameters of the network and the feature map (or image data) of the previous layer in batch units and receives the next An output feature map to a layer can be output in batch units.

The accelerator library wrapper 330 may be configured to intercept the called accelerator library function and wrap the accelerator library function to enable parallel processing by a plurality of accelerators 230_1, 230_2, ..., 230_n. . Under this structure, the code written for the single accelerator 230 at the level of the program 310 for a single accelerator and the DNN framework 320 is executed in the plurality of accelerators 230_1, 230_2, ..., 230_n at the library level. can For example, the accelerator library wrapper 330 may refer to a GPU library wrapper. In one embodiment, when a program 310 for a single accelerator using the accelerator library wrapper 330 is processed using the accelerator library wrapper 330 , the accelerator library through modification of the build process of the DNN framework 320 . Without modification of functions and source code (eg, user program and DNN framework 320), the program 310 for a single accelerator can be processed in parallel on a plurality of accelerators 230_1, 230_2, ..., 230_n. there is. For example, the processor 210 may allocate an accelerator library function to each of the plurality of accelerators 230_1 , 230_2 , ..., 230_n.

4 is a flowchart illustrating a method 400 of processing a program for a single accelerator using a DNN framework in a plurality of accelerators according to an embodiment of the present disclosure. This method 400 may be performed by a processor. As shown, the method 400 may begin with receiving a call to a deep learning computation function ( S410 ). For example, a DNN framework running on a processor may receive a call to a deep learning computation function from a program for a single accelerator.

Then, in step S420 , a call to an accelerator library function for executing a deep learning operation function in a single accelerator may be received. For example, an accelerator library wrapper running on a processor may receive calls from the DNN framework to accelerator library functions to execute deep learning computation functions on a single accelerator.

Next, in step S430 , in response to the call of the accelerator library function, an accelerator library function may be assigned to each of a plurality of accessible accelerators. In one embodiment, the accelerator library wrapper on the processor may receive a call from the DNN framework to an accelerator library function for executing a deep learning computation function on a single accelerator, and assign the accelerator library function to each of the plurality of accelerators. there is. In this case, the accelerator library wrapper may divide the input data of the accelerator library function into a plurality of partial input data sets, and assign each of the divided partial input data sets to each of the plurality of accelerators. For example, when input data of an accelerator library function is received in batch units, the accelerator library wrapper distributes the input data to each of a plurality of accelerators in batch units and divides them into one or more data sets. there is.

According to an embodiment, the accelerator library wrapper may assign each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators. Here, in assigning each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators, the memory access pattern of the accelerator library function is analyzed and based on the analyzed access pattern, before execution of the accelerator library function Input data of an accelerator library function may be assigned to each of a plurality of accessible accelerators. For example, the access patterns of inputs and outputs of DNN-related accelerator library functions to memory use compiler analysis techniques and/or profiling techniques (for example, techniques to identify input/output patterns in memory by executing functions in advance). can be analyzed. As another example, before the accelerator library function is executed, data may be pre-distributed to a plurality of accelerators using an access pattern analyzed by a compiler analysis and/or profiling technique. Due to the nature of the operation of the DNN framework, the same pattern of tasks may be repeated during multiple iterations during the entire learning or inference process. If the memory access pattern for the first or several iterations (for example, 3 times) is identified using these characteristics, the memory access pattern of each accelerator library function can be analyzed during the entire learning or inference process.

Next, in step S440 , intermediate result data of processing the accelerator library function may be received from each of the plurality of accelerators. According to an embodiment, the accelerator library wrapper may receive intermediate result data from a plurality of accelerators that have processed the accelerator library function. For example, the accelerator library wrapper may receive intermediate result data of processing each of a plurality of partial input data sets using an accelerator library function in each of the plurality of accelerators. As another example, when the output data of the accelerator library function is output in batches, the accelerator library wrapper may receive intermediate result data obtained by processing each of the input data sets from a plurality of accelerators in batches.

Finally, in step S450 , result data for the called accelerator library function may be generated based on the received intermediate result data. For example, the accelerator library wrapper may generate result data based on intermediate result data received from a plurality of accelerators, which will be described in more detail later with reference to FIGS. 7 and 8 .

A plurality of accelerators 530_1, 530_2, ..., 530_n (where n is a natural number greater than or equal to 2) illustrated in FIGS. 5 to 8, which will be described later, are included in one computing device or a cluster including a plurality of computing devices. can be included in the system. Here, the plurality of accelerators 530_1, 530_2, ..., 530_n may include accelerators accessible by a computing device (node) in which the DNN framework operates.

5 is a diagram illustrating input data 510 divided into a plurality of partial input data sets 520_1, 520_2, ..., 520_n, and divided into a plurality of partial input data sets 520_1 and 520_2 according to an embodiment of the present disclosure. , ..., 520_n) is a diagram illustrating an example of allocating each of the accelerators 530_1, 530_2, ..., 530_n.

According to an embodiment, as shown, the input data 510 of the accelerator library function may be divided into a plurality of partial input data sets 520_1, 520_2, ..., 520_n. Each of the divided plurality of partial input data sets 520_1, 520_2, ..., 520_n and a corresponding accelerator library function may be assigned to each of the plurality of accelerators 530_1, 530_2, ..., 530_n.

According to an embodiment, when the input data 510 is received in batch units, the accelerator library wrapper may equally distribute the input data 510 to the plurality of accelerators 530_1 , 530_2 , ..., 530_n. For example, in Tensorflow or Pytorch, the memory layout of data corresponding to neurons is usually "NCHW" or "NHWC", where "N" is first, so it may correspond to a layout where the highest dimension is a layout. . Accordingly, if it is assumed that there are two accelerators included in the cluster system, the entire data may be divided into two consecutive parts and distributed to the two accelerators.

In another embodiment, if the processing speed of the distributed partial input data is different or there is a difference in processing performance due to the network configuration of the cluster system, etc., the plurality of partial input data sets 520_1, 520_2, .. ., 520_n) may be differentially distributed to a plurality of accelerators.

6 is an example of allocating data when the input data 610 cannot be divided into a plurality of partial input data sets to the plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure; It is a drawing showing

In an embodiment, the accelerator library wrapper may allocate the same input data 610 to the plurality of accelerators 530_1 , 530_2 , ..., 530_n when the input data 610 cannot be divided into batch units. . Here, the case in which the input data 610 cannot be divided into batch units may be a case in which the corresponding input data 610 is required in a plurality of accelerators 530_1, 530_2, ..., 530_n, and in DNN, mainly network parameters are This may apply. For example, in Tensorflow or Pytorch, the memory layout of parameter data is usually "CKHW" or "HWKC", which is batch-independent and cannot be broken down into batches. Accordingly, a copy of the input data 610 may be allocated to each of the plurality of accelerators 530_1, 530_2, ..., 530_n so that the plurality of accelerators 530_1, 530_2, ..., 530_n use the same data. there is.

According to another embodiment, the accelerator library wrapper divides parameter data (ie, weight; not shown) of the accelerator library function into a plurality of partial parameter data sets, and divides each of the divided partial parameter data sets into a plurality of accelerators 530_1, 530_2, ..., 530_n) can be provided. In this case, the accelerator library wrapper may provide the accelerator library function associated with the divided parameter data set to the plurality of accelerators 530_1, 530_2, ..., 530_n. In addition, the accelerator library wrapper provides input data 610 of an accelerator library function associated with a partitioned parameter data set to a plurality of accelerators 530_1, 530_2, ..., 530_n with a partitioned parameter data set and an accelerator library function. can do. Here, as shown, the input data 610 may not be divided. In contrast, the accelerator library wrapper divides the input data 610 of the accelerator library function associated with the partitioned parameter data set into a plurality of partial input data sets, and divides the partitioned plurality of partial input data sets into the partitioned parameter data set and It can be provided to the plurality of accelerators 530_1, 530_2, ..., 530_n together with the accelerator library function.

7 shows intermediate result data 710_1, 710_2, ..., 710_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure by connecting result data 720 ) is a diagram showing an example of generating According to an embodiment, the accelerator library wrapper receives intermediate result data 710_1, 710_2, ..., 710_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n, and receives the received intermediate result data. By concatenating (710_1, 710_2, ..., 710_n), result data 720 may be generated. For example, in the case of the convolution function among the accelerator library functions provided by CuDNN, intermediate result data (710_1, 710_2, ..., 710_n) received from a plurality of accelerators (530_1, 530_2, ..., 530_n) is connected. By doing so, result data 720 may be generated.

In one embodiment, the accelerator library wrapper 330 is configured to convert the intermediate result data 710_1, 710_2, ..., 710_n when the intermediate result data 710_1, 710_2, ..., 710_n are parallelized in batches. Concatenate to produce result data 720 . For example, when the intermediate result data 710_1, 710_2, ..., 710_n are parallelized in batches, the intermediate result data 710_1, 710_2, ..., 710_n are simply connected as the result data 710. can be saved

8 is a diagram illustrating result data 820 by calculating intermediate result data 810_1, 810_2, ..., 810_n received from a plurality of accelerators 530_1, 530_2, ..., 530_n according to an embodiment of the present disclosure. ) is a diagram showing an example of generating As shown in FIG. 8, the accelerator library wrapper is intermediate result data (810_1, 810_2, ..., 810_n), the intermediate result data 810_1, 810_2, ..., 810_n obtained by processing the parameter data of the accelerator library function may be calculated to generate the result data 820 . This computational process may be referred to as reduction. For example, in the case of max pooling among the accelerator library functions provided by CuDNN, intermediate result data (810_1, 810_2, ..., 810_n) received from a plurality of accelerators (530_1, 530_2, ..., 530_n) is calculated. By processing, result data 820 may be generated.

According to an embodiment, when the intermediate result data 810_1 , 810_2 , ..., 810_n are not parallelized in the arrangement direction, an operation for generating gradient data of network parameters necessary for the learning process may be used. A plurality of accelerators (530_1, 530_2, ..., 530_n) can calculate a network parameter gradient from the input data distributed to them, and exchange the intermediate result data (810_1, 810_2, ..., 810_n) to sum them. there is.

9 is an example of performing an operation in a plurality of accelerators (930_1, ..., 930_n; where n is a natural number equal to or greater than 2) according to an embodiment of the present disclosure and copying a plurality of partial input data sets at the same time; It is a drawing showing According to an embodiment, when the partial input data set 910 allocated to each of the plurality of accelerators 930_1, ..., 930_n includes n partial input data sets (where n is a natural number of 2 or more) ), among the n partial input data sets 910, the mth partial input data set 920_1 is processed by each of the plurality of accelerators 930_1, ..., 930_n, while the m+1th partial input data set ( The method may include allocating 920_2 to each of the plurality of accelerators 930_1, ..., 930_n (where m is a natural number smaller than n). To this end, before allocating each of the plurality of partial input data sets 910 to each of the plurality of accelerators 930_1 , ..., 930_n, the memory access pattern of the accelerator library function to be processed may be analyzed.

In one embodiment, the accelerator library wrapper 330 allocates the mth partial input data set 920_1, which is a part of the plurality of partial input data sets 910, to the accelerator 930_1 based on the analyzed memory access pattern. can be processed While the m-th partial input data set 920_1 is processed by the accelerator 930_1 , the accelerator library wrapper 330 may allocate the m+1-th partial input data set 920_2 to the accelerator 930_1 . For example, due to the mechanical characteristics of the plurality of accelerators (930_1, ..., 930_n), the copy operation and operation operation of the memory in the accelerator are processed separately, so the plurality of accelerators (930_1, .. ., 930_n), it is possible to reduce the total execution time of the deep learning application by processing the copy of the memory while the operation is performed. Under this operation method, as shown, a plurality of partial input data sets 920_1 and 920_3 are allocated to each of the plurality of accelerators 930_1 and 930_n, and the allocated plurality of partial input data sets 920_1 and 920_3 are During processing, another plurality of partial input data sets 920_2 and 920_4 may be assigned to each of the plurality of accelerators 930_1 and 930_n.

The method of processing a program for a single accelerator using the above-described DNN framework in a plurality of accelerators may be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected through a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the art to which the present invention pertains.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those of ordinary skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure are suitable for use in general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the present disclosure. It may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementations, the techniques may include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on computer readable media such as programmable read-only memory), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. It may be implemented as stored instructions. The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

If implemented in software, the techniques may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer-readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. may include any other medium that can be used for transport or storage to a computer and can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, wireless, and microwave, the coaxial cable , fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. As used herein, disk and disk include CD, laser disk, optical disk, digital versatile disc (DVD), floppy disk, and Blu-ray disk, where disks are usually magnetic Data is reproduced optically, while discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, or write information to, the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and the storage medium may exist as separate components in the user terminal.

Although the embodiments described above have been described utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not so limited and may be implemented in connection with any computing environment, such as a network or distributed computing environment. . Still further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across the plurality of devices. Such devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in connection with some embodiments herein, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art to which the present disclosure pertains. Further, such modifications and variations are intended to fall within the scope of the claims appended hereto.

100: cluster system
110: network
120: host computing device
130, 130_1, 130_2, ..., 130_n: computing device
200: computing device
210: processor
220: main memory
230, 230_1, 230_2, ..., 230_n: accelerator
310: program for single accelerator
320: DNN Framework
330: accelerator library wrapper
340: accelerator library

Claims (11)

In a method for processing a program for a single accelerator using a DNN framework in a plurality of accelerators,
receiving a call to a deep learning computation function;
receiving a call to an accelerator library function for executing the deep learning computation function in a single accelerator;
in response to the invocation of the accelerator library function, assigning the accelerator library function to each of a plurality of accessible accelerators;
receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators; and
generating result data for the called accelerator library function based on the received intermediate result data;
Receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators comprises receiving intermediate result data of processing parameter data of the accelerator library function from each of the plurality of accelerators,
The generating result data for the called accelerator library function includes generating result data for intermediate result data obtained by processing parameter data of the accelerator library function,
How to process a program for a single accelerator on multiple accelerators.
In a method for processing a program for a single accelerator using a DNN framework in a plurality of accelerators,
receiving a call to a deep learning computation function;
receiving a call to an accelerator library function for executing the deep learning computation function in a single accelerator;
in response to the invocation of the accelerator library function, assigning the accelerator library function to each of a plurality of accessible accelerators;
receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators; and
generating result data for the called accelerator library function based on the received intermediate result data;
Allocating the accelerator library function to each of the plurality of accessible accelerators comprises:
dividing the input data of the accelerator library function into a plurality of partial input data sets; and
assigning each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators;
including,
Receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators may include processing each of the plurality of partial input data sets using the accelerator library function in each of the plurality of accelerators. receiving result data;
How to process a program for a single accelerator on multiple accelerators.
3. The method of claim 2,
Allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators comprises:
analyzing a memory access pattern of the accelerator library function; and
allocating input data of the accelerator library function to each of the plurality of accessible accelerators prior to execution of the accelerator library function based on the analyzed access pattern;
How to process a program for a single accelerator on multiple accelerators.
3. The method of claim 2,
Allocating each of the accelerator library function and the divided plurality of partial input data sets to each of the plurality of accelerators comprises:
When the partial input data set allocated to each of the plurality of accelerators includes n partial input data sets (where n is a natural number greater than or equal to 2), an mth partial input data set among the n partial input data sets processing in each of the plurality of accelerators while assigning an m+1th partial input data set to each of the plurality of accelerators, wherein m is a natural number less than n,
How to process a program for a single accelerator on multiple accelerators.
delete According to claim 1,
Receiving intermediate result data of processing the accelerator library function from each of the plurality of accelerators comprises receiving intermediate result data of processing parameter data of the accelerator library function from each of the plurality of accelerators,
The generating result data for the called accelerator library function includes generating result data for intermediate result data obtained by processing parameter data of the accelerator library function,
How to process a program for a single accelerator on multiple accelerators.
According to claim 1,
Based on the received intermediate result data, generating result data for the called accelerator library function comprises concatenating the received intermediate result data to generate the result data.
How to process a program for a single accelerator on multiple accelerators.
According to claim 1,
Based on the received intermediate result data, generating result data for the called accelerator library function includes calculating the received intermediate result data to generate the result data,
How to process a program for a single accelerator on multiple accelerators.
According to claim 1,
The plurality of accelerators are included in one computing device,
How to process a program for a single accelerator on multiple accelerators.
According to claim 1,
The plurality of accelerators are included in a cluster system including a plurality of computing devices,
How to process a program for a single accelerator on multiple accelerators.
A method of processing a single accelerator program using the DNN framework according to any one of claims 1 to 4 and 6 to 10 in a plurality of accelerators is stored in a computer-readable recording medium for execution in a computer. stored computer programs.

Images