KR102490539B1 - Method for generating program for use in accelerator for deep learning - Google Patents

Abstract

The present disclosure relates to a method of generating a program for an accelerator for deep learning. A method for generating a program for an accelerator for deep learning includes receiving a deep learning application, generating a unit operation list included in the deep learning application, generating an intermediate expression from the unit operation list, and based on the intermediate expression. and generating programs for accelerators for deep learning applications.

Description

Method for generating program for accelerator for deep learning {METHOD FOR GENERATING PROGRAM FOR USE IN ACCELERATOR FOR DEEP LEARNING}

The present invention relates to a method for generating a program for an accelerator for deep learning. Specifically, it relates to a method of generating an optimal accelerator program for deep learning by combining unit operations included in a deep learning application.

Application programmers who write deep learning applications usually use a deep learning framework (e.g. TensorFlow) to express combinations of unit operations. It can be mapped and executed. Individual pre-implemented programs for unit operations can be implemented using primitive libraries distributed by hardware manufacturers (e.g., Nvidia's cuDNN, Intel's MKL-DNN), and the source code of the corresponding program is not normally open to the public. not.

When processing a combination of unit operations, performance can be improved when the combination of unit operations is implemented as a single program rather than executing each unit operation as a separate program. For example, if two unit operations A and B are executed sequentially and the result of A's operation is used as an input to B, if a program is written that processes A and B simultaneously, the result of A is stored in memory and You can get rid of the overhead of reading it in B. This technique is called kernel fusion.

Research/inventions/programs to accelerate deep learning applications using kernel fusion have already been developed, but there is a limitation in that kernel fusion can be executed only for predetermined patterns. That is, according to the prior art, a program combining unit operations may not be generated for a unit operation pattern that is not previously defined.

The present disclosure provides a method and a computer program for generating a program for an accelerator for deep learning to solve the above problems.

The present disclosure may be implemented in a variety of ways, including a method, apparatus, or computer program stored on a computer readable storage medium.

According to an embodiment of the present disclosure, a method for generating a program for an accelerator for deep learning includes receiving a deep learning application, generating a unit operation list included in the deep learning application, and an intermediate expression from the unit operation list. and generating a program for an accelerator for a deep learning application based on the intermediate expression.

According to an embodiment of the present disclosure, the method further includes extracting a plurality of unit operations included in the deep learning application by analyzing the deep learning application using a unit operation division scheduler. Generating a unit operation list included in the deep learning application includes generating a unit operation list including a plurality of extracted unit operations.

According to an embodiment of the present disclosure, determining a data dependency relationship of a plurality of unit operations by analyzing a pattern of a plurality of extracted unit operations, and based on the determined data dependency relationship of a plurality of unit operations, a deep learning application and determining a plurality of corresponding division schedules.

According to an embodiment of the present disclosure, the method further includes selecting one division schedule from among a plurality of division schedules by a unit operation division scheduler. The step of generating an intermediate expression from the unit operation list is the step of extracting a plurality of unit operation intermediate expressions corresponding to the unit operation list from the intermediate expression DB based on one selected division schedule - of the plurality of unit operation intermediate expressions Each represents a data flow graph for an operation of one thread block - and generating one or more intermediate expressions by connecting a plurality of unit operation intermediate expressions extracted based on a selected split schedule.

According to an embodiment of the present disclosure, generating an accelerator program for a deep learning application based on an intermediate expression includes an accelerator program corresponding to each of one or more intermediate expressions generated based on one selected split schedule. It includes the step of generating.

According to an embodiment of the present disclosure, determining the performance of the generated accelerator program, providing the determined performance of the accelerator program to a unit operation division scheduler, and dividing unit operations based on the determined performance of the accelerator program The method further includes selecting, by the scheduler, another divided schedule other than the selected divided schedule from among the plurality of divided schedules.

According to an embodiment of the present disclosure, the method further includes generating programs for a plurality of accelerators corresponding to the plurality of determined split schedules and determining performance of each of the programs for the plurality of accelerators. Generating an accelerator program for a deep learning application based on the intermediate expression includes selecting one accelerator program from among a plurality of generated accelerator programs based on the determined performance.

According to an embodiment of the present disclosure, the intermediate representation is represented as a graph including a plurality of nodes representing each scalar operation and an edge representing a dependency relationship between the scalar operations. The step of generating a program for an accelerator for a deep learning application based on the intermediate expression includes generating a key operation code corresponding to an operation of each node by traversing the graph of the intermediate expression in reverse postorder traversal.

According to an embodiment of the present disclosure, generating a program for an accelerator for a deep learning application based on an intermediate expression includes acquiring hardware and a target program language used in the deep learning application, from a pre-stored basic code DB. , extracting a base code corresponding to the acquired hardware and target program language, and generating a program for an accelerator including the extracted base code and the generated core operation code.

According to one embodiment of the present disclosure, a computer program stored in a computer-readable recording medium is provided to execute the above-described method for generating a program for an accelerator for deep learning in a computer.

According to some embodiments of the present disclosure, the processor generates in real time an accelerator program corresponding to a combination of unit operations capable of exhibiting optimal performance based on unit operations included in a deep learning application, thereby reducing overhead and the like. This can improve the performance of deep learning applications.

According to some embodiments of the present disclosure, the processor may generate a program for an accelerator by combining unit operations constituting the deep learning application, even if the deep learning application includes any unit operation and / or combination of unit operations, Based on the performance of each accelerator program according to the division schedule, it is possible to generate and/or select an accelerator program having optimal performance.

According to some embodiments of the present disclosure, a processor can generate a program for an accelerator in real time by combining one or more unit operations without the intervention of a programmer, so a series of processes for generating an optimal program for an accelerator can be automated. . That is, code for deep learning applications can be generated or the performance of existing deep learning applications can be improved without additional human resource input.

According to some embodiments of the present disclosure, the processor may generate an optimal intermediate expression by appropriately connecting one or more unit operation intermediate expressions according to a division schedule.

According to some embodiments of the present disclosure, a processor may generate a program for an accelerator by generating codes corresponding to intermediate expressions in real time.

According to some embodiments of the present disclosure, a final accelerator program having the best performance among accelerator programs according to a division schedule may be generated, and the created final accelerator program may be used to drive a deep learning application.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein like reference numbers indicate like elements, but are not limited thereto.
1 is a diagram illustrating an example in which a processor generates a program for an accelerator according to an embodiment of the present disclosure.
2 is a block diagram showing an internal configuration of a processor according to an embodiment of the present disclosure.
3 is a flowchart illustrating an example of a method of generating a program for a deep learning-based accelerator according to an embodiment of the present disclosure.
4 is a diagram illustrating an example of generating a unit operation list by a unit operation division scheduler according to an embodiment of the present disclosure.
5 is a diagram illustrating an example in which a unit operation division scheduler determines a plurality of division schedules according to an embodiment of the present disclosure.
6 is a diagram illustrating an example of generating an intermediate expression by an intermediate expression conversion module according to an embodiment of the present disclosure.
7 is a diagram illustrating an example in which an intermediate expression conversion module connects unit operation intermediate expressions according to an embodiment of the present disclosure.
8 is a diagram illustrating an example in which a code generator generates a program for an accelerator according to an embodiment of the present disclosure.
9 is an exemplary diagram illustrating a dataflow graph of an intermediate expression according to an embodiment of the present disclosure.
10 is a block diagram illustrating an example of generating a program for a final accelerator according to an embodiment of the present disclosure.

Hereinafter, specific details for the implementation of the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding elements are given the same reference numerals. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, omission of a description of a component does not intend that such a component is not included in an embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the following embodiments in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, but only the present embodiments make the present disclosure complete, and the present disclosure does not extend the scope of the invention to those skilled in the art. It is provided only for complete information.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary according to the intention of a person skilled in the related field, a precedent, or the emergence of new technology. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the general content of the present disclosure, not simply the names of the terms.

Expressions in the singular number in this specification include plural expressions unless the context clearly dictates that they are singular. Also, plural expressions include singular expressions unless the context clearly specifies that they are plural. When it is said that a certain part includes a certain component in the entire specification, this means that it may further include other components without excluding other components unless otherwise stated.

Also, the term 'module' or 'unit' used in the specification means a software or hardware component, and the 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a 'module' or 'unit' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, and attributes. , procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. Functions provided within components and 'modules' or 'units' may be combined into a smaller number of components and 'modules' or 'units', or further components and 'modules' or 'units'. can be further separated.

According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented with a processor and a memory. 'Processor' should be interpreted broadly to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, accelerators such as multicore CPUs, GPUs, FPGAs, etc. . In some circumstances, 'processor' may refer to an application specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or a combination of any other such configurations. You may. Also, 'memory' should be interpreted broadly to include any electronic component capable of storing electronic information. 'Memory' includes random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable-programmable read-only memory (EPROM), It may also refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor can read information from and/or write information to the memory. Memory integrated with the processor is in electronic communication with the processor.

In the present disclosure, a 'deep neural network (DNN) framework' or a 'deep learning framework' may include any created software assembly that facilitates the creation and execution of deep learning applications. According to one embodiment, such a DNN framework applies a deep learning processing or deep learning operation function to an accelerator so that a developer can more easily use a parallel programming model or program requiring high proficiency, thereby performing a training process and It can speed up the inference process. For example, DNN frameworks may include, but are not limited to, recently widely used DNN frameworks such as Caffe, Tensorflow, Pytorch, CNTK, and Theano.

In the present disclosure, 'deep learning application' may refer to a combination of a plurality of unit operations. According to one embodiment, a deep learning application programmer can create a deep learning application by writing a combination of a plurality of unit operations using a deep learning framework (eg, Tensorflow, Pytorch, CNTK, Caffe, etc.) .

In the present disclosure, an 'element-wise operation' may refer to any scalar operation and/or a set of scalar operations constituting a deep learning application, but is not limited thereto, for example, an addition operation, Subtraction operation, maximum value operation, minimum value operation, floating point multiplication operation, convolution, matrix multiplication, Recitified Linear Unit (ReLU), pooling, Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU) ) and the like.

In the present disclosure, 'unit operation intermediate expression' may refer to a Data Flow Graph (DFS) representing an operation of one thread block. For example, the unit operation intermediate expression may be expressed as a graph including a plurality of nodes representing scalar operations and edges representing dependencies between scalar operations.

In the present disclosure, 'intermediate expression' may refer to a combination of one or more unit operation intermediate expressions generated based on data dependency relationships, and may be represented by, for example, DFS.

In the present disclosure, 'division schedule' may refer to a schedule for how to implement a program for an accelerator by analyzing a pattern of unit operations included in a deep learning application, One or more division schedules or a plurality of division schedules may be determined based on an execution order, a dependency relationship of unit operations, and the like.

In the present disclosure, a 'core operation code' may refer to a code constituting at least a part of a program for an accelerator, and may refer to a code representing an intermediate expression, for example. According to one embodiment, when an accelerator is used as a GPU and a CUDA program is used as a target for the GPU, the core operation code may refer to an operation part in the middle of the kernel of the accelerator program.

In the present disclosure, an 'accelerator program' refers to any code and/or executable file that can operate on an accelerator, and may include a C program, OpenCL, or CUDA program.

In the present disclosure, 'code' may refer to any code prepared to execute a program, and may refer to, for example, source code.

1 is a diagram illustrating an example in which a processor 100 generates a program 120 for an accelerator according to an embodiment of the present disclosure. For example, the accelerator program 120 represents arbitrary codes and/or executable files operable on the accelerator, and may include programs operable in a multi-core CPU, GPU, FPGA, and the like. Also, the accelerator program 120 may refer to an arbitrary program generated by combining/combining one or more unit operations. Here, the unit operation may refer to any operation that can be used in deep learning, and for example, convolution, matrix multiplication, Recitified Linear Unit (ReLU), pooling, long short-short-term (LSTM) Term Memory) and GRU (Gated Recurrent Unit) operations.

According to one embodiment, the processor 100 may receive the deep learning application 110 . Here, the deep learning application 110 may refer to any software program composed or expressed as a combination of unit operations or information and/or data for generating any software program. For example, the deep learning application 110 may be generated by arbitrarily combining or connecting unit operations such as convolution, matrix multiplication, ReLU, pooling, LSTM, and GRU described above. The processor 100 may extract unit operations included in the deep learning application 110 from the received deep learning application 110 and generate a unit operation list including the extracted unit operations. In this case, the unit operation list may include each unit operation included in the deep learning application 110, an execution order of each unit operation, and a dependence of each unit operation.

The processor 100 may then generate an intermediate representation from the unitary operation list. For example, the intermediate expression is for generating a code that is the basis of the accelerator program 120, and may be generated using a unit operation intermediate expression. Here, the unit operation intermediate expression may refer to a data flow graph (DFG) of unit operation. For example, the dataflow graph may include nodes representing scalar operations and edges representing data dependency relationships between scalar operations. Accordingly, an intermediate representation may refer to a dataflow graph created by combining dataflow graphs of these unitary operations. According to an embodiment, the processor 100 may receive one or more unit operation intermediate expressions from an intermediate expression database (DB). For example, the processor 100 may generate an intermediate expression by concatenating one or more unit operation intermediate expressions, performing shared memory optimization, and distributing operations among threads.

The processor 100 may generate an accelerator program 120 for the deep learning application 110 based on the intermediate expression. According to one embodiment, the processor 100 may generate code for the accelerator program 120 based on the intermediate expression. In this case, the processor 100 may generate and/or determine one accelerator program 120 or generate and/or determine two or more accelerator programs 120 according to a combination of unit operations exhibiting optimal performance. there is. With this configuration, the processor 100 generates an accelerator program corresponding to a combination of unit operations capable of exhibiting optimal performance based on the unit operations included in the deep learning application, thereby reducing overhead and deep learning. It can improve the performance of running applications. For example, the processor 100 may generate such an accelerator program in real time.

2 is a block diagram showing an internal configuration of a processor 100 according to an embodiment of the present disclosure. As shown, the processor 100 may include a unit operation division scheduler 210, an intermediate expression conversion module 220, a code generator 230, a performance measurer 240, and the like. In addition, the processor 100 may exchange information and/or data necessary for generating an accelerator program through communication with the basic code DB 250 and the intermediate expression DB 260.

As discussed above, the processor 100 may receive a deep learning application. For example, a deep learning application may include a plurality of unit operations, an execution order of each unit operation, a dependency relationship between each unit operation, and the like. In another example, when the deep learning application includes only a plurality of unit operations, the processor 100 and/or the unit operation division scheduler 210 analyzes a pattern of the plurality of unit operations of the deep learning application to determine the plurality of unit operations. Data dependencies can be determined.

According to an embodiment, the unit operation division scheduler 210 may determine a schedule for grouping a plurality of unit operations based on information included in the deep learning application. That is, the unit operation division scheduler 210 may determine a plurality of division schedules corresponding to the deep learning application based on the data dependency relationship of the plurality of unit operations. For example, if a deep learning application contains three unit operations such as A, B, and C, and A, B, and C are executed sequentially and there is a data dependency between A-B and B-C, the unit operation is divided. The scheduler 210 may generate a total of four divided schedules of (A, B, C), (A-B, C), (A, B-C), and (A-B-C).

According to an embodiment, the unit operation division scheduler 210 may generate all possible division schedules based on a plurality of unit operations, an execution order of each unit operation, a dependency relationship between each unit operation, and the like. Additionally or alternatively, the unit operation division scheduler 210 may not generate a division schedule recognized in advance as having low performance among possible division schedules, or may exclude it from the generated division schedule. For example, it can be assumed that a deep learning application includes three unit operations such as A, B, and C, that A, B, and C are executed sequentially, and that there is a data dependency relationship between A-B and B-C. In this case, if it is confirmed that configuring A and B as separate accelerator programs has better performance than configuring A-B as one program, the possibility of improving performance by configuring A-B-C as one program is there may be few In this case, it is treated as an unnecessary division schedule, and search such as program creation and performance measurement may not proceed.

According to an embodiment, the processor 100 and/or the unit operation division scheduler 210 extracts a plurality of unit operations included in the deep learning application by analyzing the deep learning application, and includes the plurality of extracted unit operations. You can create a list of unit operations that Here, the unit operation list may include a division schedule calculated by the unit operation division scheduler 210 .

The processor 100 may transmit the generated unit operation list to the intermediate expression DB 260 . Then, the processor 100 may receive each unit operation intermediate expression included in the unit operation list and/or a combination of unit operation intermediate expressions according to a division schedule from the intermediate expression DB 260 . Here, the unit operation intermediate expression is expressed in the form of a dataflow graph including nodes representing scalar operations (eg, floating point multiplication, etc.) and edges representing dependencies between scalar operations. It can be.

According to an embodiment, the intermediate expression conversion module 220 may generate an intermediate expression by using a plurality of unit operation intermediate expressions received from the intermediate expression DB 260 . That is, the intermediate expression conversion module 220 may generate an intermediate expression by connecting the unit operation intermediate expressions based on the execution order and/or data dependency relationship of the unit operations associated with the deep learning application. Then, the intermediate expression conversion module 220 may perform operation distribution between threads to determine which operation each thread will process. For example, the intermediate expression conversion module 220 may perform operation distribution between threads to improve performance of a deep learning application by using a predetermined algorithm or the like.

According to one embodiment, the intermediate expression conversion module 220 directly connects each unit operation intermediate expression, or stores the result of one unit operation intermediate expression in a shared memory, and then stores another unit operation intermediate expression in the shared memory. You can also connect to read the result. For example, when connecting the unit operation intermediate expressions of A and B, the intermediate expression conversion module 220 may directly connect a node of A (a node corresponding to a scalar operation) and a node of B. In another example, the intermediate expression conversion module 220 may store the result of the unit operation intermediate expression A in the shared memory, and connect the unit operation intermediate expression such that the stored result is read by the B unit operation intermediate expression. The method of connecting the unit operation intermediate expression may be determined based on the performance of the accelerator program, and when the result of the unit operation intermediate expression is stored in the shared memory, shared memory optimization is performed based on a predetermined algorithm or the like. It can be.

According to one embodiment, the code generator 230 may generate code for an accelerator program using the intermediate expression generated by the intermediate expression conversion module 220 . That is, the code generator 230 may generate a core operation code to perform an operation corresponding to the intermediate expression based on an intermediate expression generated by combining and/or combining one or more unit operation intermediate expressions. Additionally, code generator 230 may generate and/or extract base code for executing core operation codes. Such a base code may be a skeleton code determined based on hardware used in a deep learning application and a target programming language. That is, the code generator 230 may generate a program for an accelerator using the generated core operation code and basic code.

According to one embodiment, the code generator 230 acquires hardware and target program language used for deep learning applications, and obtains basic codes corresponding to the acquired hardware and target program language from the pre-stored basic code DB 250. can be extracted. In this case, the code generator 230 may use the base code extracted from the base code DB 250 as it is or generate a new base code using the base code. For example, the base code includes host program code for allocating memory and transmitting data to execute device functions (eg, CUDA device functions), host program code for calling kernels, header codes for device functions, It may include device function codes that are executed independently of core operations (eg, code related to thread ID acquisition and index initialization), codes for reading data necessary for core operation execution from global memory, and the like.

According to one embodiment, the performance measurer 240 may measure the performance of the accelerator program generated by the code generator 230 . Here, the performance measurer 240 may measure the performance of the accelerator program using performance indicators of execution time, energy consumption, and/or resource usage, but is not limited thereto, and of the deep learning application (or accelerator program). Any performance metric that can measure performance may be used. In this case, the performance measurer 240 actually runs the accelerator program to measure the performance of the accelerator program, or uses a predetermined algorithm to measure the performance of the deep learning application (or accelerator program) to measure the performance of the accelerator program. Performance can also be measured. For example, the performance measurer 240 may input test data into the generated accelerator program and measure an execution time when a result value corresponding to the test data is produced. At this time, when the execution time is short with an expected result value, it may be determined that the performance is high.

According to an embodiment, the performance measurer 240 may measure the performance of each accelerator program generated according to a plurality of division schedules. Accordingly, the processor 100 may compare performance of each accelerator program according to a plurality of division schedules and select an accelerator program with the best performance. With this configuration, the processor 100 may generate a program for an accelerator by combining unit operations constituting the deep learning application, even if the deep learning application includes any unit operation and / or combination of unit operations, Based on the performance of each accelerator program according to the division schedule, it is possible to generate and/or select an accelerator program having optimal performance.

The base code DB 250 and the intermediate expression DB 260 may refer to databases connected to the processor 100 or accessible through communication. In FIG. 2 , the base code DB 250 and the intermediate expression DB 260 are illustrated to be configured as separate databases, but are not limited thereto and may be configured as one database.

In addition, in FIG. 2, the configuration of the processor 100 has been described separately for each function, but this does not necessarily mean that they are physically separated. In FIG. 2, the unit operation division scheduler 210 and the intermediate expression conversion module 220 have been separately described, but this is only to help understanding of the present invention, and one arithmetic unit may perform two or more functions. With this configuration, since the processor 100 can generate an accelerator program in real time by combining one or more unit operations without a programmer's intervention, a series of processes for generating an optimal accelerator program can be automated. . That is, code for deep learning applications can be generated or the performance of existing deep learning applications can be improved without additional human resource input.

3 is a flowchart illustrating an example of a method 300 of generating a program for an accelerator for deep learning according to an embodiment of the present disclosure. According to one embodiment, the method 300 of generating a program for an accelerator for deep learning may be performed by at least one processor (eg, the processor of FIG. 2 ). The method 300 for generating a program for an accelerator for deep learning may be initiated when a processor receives a deep learning application (S310). Here, the deep learning application may include one or more scalar operations and/or one or more unit operations.

According to one embodiment, the processor may generate a unit operation list included in the deep learning application (S320). In this case, the processor may extract a plurality of unit operations included in the deep learning application by analyzing the deep learning application using a unit operation division scheduler, and generate a unit operation list including the plurality of extracted unit operations. .

Then, the processor may generate an intermediate expression from the unit operation list (S330). For example, the processor may provide a unit operation list to an intermediate expression DB, and extract one or more unit operation intermediate expressions corresponding to the unit operation list from the intermediate expression DB. An intermediate expression may be generated by concatenating one or more extracted unit operation intermediate expressions.

The processor may generate a program for an accelerator for a deep learning application based on the intermediate expression (S340). Here, the intermediate expression may be expressed as a dataflow graph including a plurality of nodes representing each scalar operation and an edge representing a dependency relationship between the scalar operations. In this case, the processor may generate a key operation code corresponding to the operation of each node by searching the graph of the intermediate expression in reverse postorder. According to one embodiment, the processor may acquire hardware and target program language used for deep learning applications. In addition, the processor may extract a basic code corresponding to the obtained hardware and a target program language from a pre-stored basic code DB, and generate a program for an accelerator including the extracted basic code and core operation code.

4 is a diagram illustrating an example of generating a unit operation list 420 by the unit operation division scheduler 210 according to an embodiment of the present disclosure. As shown, the unit operation division scheduler 210 may generate the unit operation list 420 using the deep learning application 410 . According to one embodiment, the deep learning application 410 may include one or more unit operations, and each unit operation may include one or more scalar operations. According to another embodiment, the deep learning application 410 may include one or more unit operations, an execution order of each unit operation, a dependency relationship between each unit operation, and the like.

The unit operation division scheduler 210 may extract a plurality of unit operations included in the deep learning application 410 by analyzing the deep learning application 410 . In this case, the unit operation division scheduler 210 may determine a data dependency relationship of the plurality of unit operations by analyzing patterns of the plurality of unit operations of the deep learning application 410 . Also, the unit operation division scheduler 210 may determine an execution order of a plurality of unit operations by analyzing a pattern of a plurality of unit operations of the deep learning application 410 .

According to an embodiment, the unit operation division scheduler 210 may generate a unit operation list 420 including a plurality of extracted unit operations. In this case, the unit operation division scheduler 210 may determine a plurality of division schedules corresponding to the deep learning application 410 based on the determined data dependency relationship of the plurality of unit operations and/or the execution order of the plurality of unit operations. there is. Then, the unit operation division scheduler 210 may generate a unit operation list 420 corresponding to each division schedule. That is, a plurality of unit operation lists 420 corresponding to one deep learning application 410 may be generated according to a plurality of division schedules.

5 is a diagram illustrating an example in which the unit operation division scheduler 210 determines a plurality of division schedules 520, 530, 540, and 550 according to an embodiment of the present disclosure. According to one embodiment, the plurality of unit operations 510 (eg, A->B->C) are extracted from a deep learning application or determined by the unit operation division scheduler 210, and the type of unit operation , execution order, dependencies, etc. Here, the plurality of unit operations 510 are composed of A, B, and C unit operations, have an execution order of A->B->C, and may have a data dependency relationship between A-B and B-C.

According to an embodiment, the unit operation division scheduler 210 may determine four division schedules 520 , 530 , 540 , and 550 from the unit operation 510 . For example, splitting schedule 520 for generating A, B, and C unit operations into respective intermediate representations, generating A and B unit operations into one intermediate representation, and C unit operations into another intermediate representation. A split schedule 530 for generating, B and C unit operations as one intermediate representation, a split schedule 540 for generating A unit operations as another intermediate representation, and A, B and C unit operations A division schedule 550 for generating each of them into one intermediate representation may be determined.

According to an embodiment, the unit operation division scheduler 210 generates a unit operation list corresponding to each division schedule 520, 530, 540, and 550, and intermediate unit operation intermediate expressions corresponding to the generated unit operation list. Can be extracted from expression DB. For example, when a unit operation list corresponding to the division schedule 520 is generated, the unit operation division scheduler 210 may extract unit operation intermediate expressions of A, B, and C, respectively, from the intermediate expression DB. In another example, when the unit operation list corresponding to the division schedule 530 is generated, the unit operation division scheduler 210 obtains a unit operation intermediate expression in which A and B are combined and a unit operation intermediate expression of C from the intermediate expression DB. or the intermediate expression of unit operations of A, B, and C can be extracted.

6 is a diagram illustrating an example of generating an intermediate expression 620 by the intermediate expression conversion module 220 according to an embodiment of the present disclosure. As described above, the processor and/or the unit operation division scheduler may extract the unit operation intermediate expression 610 corresponding to the unit operation list from the intermediate expression DB. Here, the unit operation intermediate expression 610 may be composed of n (n is a natural number) unit operation intermediate expressions extracted based on the unit operations included in the unit operation list, the execution order of the unit operations, and dependency relationships. The unit operation intermediate expression 610 extracted in this way may be provided to the intermediate expression conversion module 220 .

The intermediate expression conversion module 220 may generate the intermediate expression 620 using the unit operation intermediate expression 610 . According to one embodiment, the intermediate expression conversion module 220 connects a plurality of unit operation intermediate expressions 610 extracted based on one division schedule selected from among a plurality of division schedules, thereby generating One or more intermediate representations 620 may be created. For example, when intermediate expressions of unit operations of A, B, and C are extracted based on a split schedule of A-B-C, the intermediate expression conversion module 220 converts A, B, and C according to an execution order and/or dependency relationship. By concatenating, one intermediate expression can be created. For example, if the division schedule is (A-B, C) and A, B, and C are extracted as unit operation intermediate expressions 620, the intermediate expression conversion module 220 connects unit operation intermediate expressions A and B can do what you do.

According to an embodiment, the operations included in the intermediate expression 620 may be divided/distributed by determining which operation to be processed by a plurality of threads, respectively. For example, the intermediate expression conversion module 220 may perform operation distribution between threads to maximize performance of a deep learning application by using a predetermined algorithm or the like. In addition, when some operations included in the intermediate expression 620 are performed and the result is stored on the shared memory, the intermediate expression conversion module 220 may perform shared memory optimization based on a predetermined algorithm or the like. .

7 is a diagram illustrating an example in which the intermediate expression conversion module 220 connects unit operation intermediate expressions 710 and 720 according to an embodiment of the present disclosure. For example, A unitary operation intermediate representation 710 may consist of three a scalar operations and two b scalar operations, and B unitary operation intermediate representation 720 may consist of three c scalar operations and one d scalar operation. may consist of Here, the scalar operation may represent an addition operation, a subtraction operation, a maximum value operation, a minimum value operation, a floating point multiplication operation, and the like, and a unit operation having one or more scalar operations as a set is used to create a deep learning application It may represent convolution, matrix multiplication, Recitified Linear Unit (ReLU), pooling, Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and the like.

The intermediate expression conversion module 220 may generate an intermediate expression 730 by concatenating the unit operation intermediate expression 710 and the unit operation intermediate expression 720 . That is, the intermediate expression 730 may be a set of unit operation intermediate expressions 710 and 720 for generating one or more unit operations into one accelerator program. The intermediate expression transformation module 220 may connect the unit operation intermediate expression 710 and the unit operation intermediate expression 720 in consideration of the division schedule. In other words, the unit operation intermediate expression 710 and the unit operation intermediate expression 720 may be connected based on the execution order and dependency relationship of the unit operation intermediate expression 710 and the unit operation intermediate expression 720 .

In the illustrated example, unit operation intermediate expression 710 may be executed first, followed by unit operation intermediate expression 720 . In addition, the result value of the b scalar operation of the unit operation intermediate expression 710 may be used as an input of the c scalar operation of the unit operation intermediate expression 720 . In this case, the intermediate expression conversion module 220 may generate the intermediate expression 730 by directly connecting the b scalar operation of the unit operation intermediate expression 710 and the c scalar operation of the unit operation intermediate expression 720 .

In FIG. 7 , the intermediate expression conversion module 220 is shown as generating an intermediate expression 730 by directly connecting the scalar operation of the unit operation intermediate expression 710 and the scalar operation of the unit operation intermediate expression 720, but is limited thereto. It doesn't work. For example, the intermediate expression conversion module 220 may store the execution result of the unit operation intermediate expression 710 on the shared memory in consideration of shared memory optimization and/or operation distribution between threads, and in this case, the unit operation intermediate Representation 710 may be created as a separate intermediate representation. In addition, the intermediate expression conversion module 220 generates the unit operation intermediate expression 720 as a separate intermediate expression, and the unit operation intermediate expression 720 converts the execution result of the unit operation intermediate expression 710 stored in the shared memory. you can have it read. With this configuration, the intermediate expression transformation module 220 can generate an optimal intermediate expression by appropriately connecting one or more unit operation intermediate expressions according to a division schedule.

8 is a diagram illustrating an example in which a code generator 230 generates a program 820 for an accelerator according to an embodiment of the present disclosure. As described above, the code generator 230 may use the intermediate representation 810 and the base code 830 to generate the program 820 for the accelerator. Here, the intermediate expression 810 can be represented as a dataflow graph including a plurality of nodes each representing a scalar operation and an edge representing a dependency relationship between the scalar operations.

According to one embodiment, the code generator 230 may generate a program 820 for an accelerator for a deep learning application based on the intermediate expression 810 . In this case, the code generator 230 may search the graph of the intermediate expression 810 in reverse postorder to generate a key operation code corresponding to the operation of each node, but is not limited thereto. Here, the core operation code may be a code part for executing an operation corresponding to the intermediate expression 810 in the kernel of the accelerator program 820 .

According to an embodiment, the code generator 230 may generate a program 820 for an accelerator for a deep learning application based on the intermediate expression 810 and the base code 830 . The code generator 230 may obtain hardware and target program language used for deep learning applications. Then, it is possible to extract the base code 830 corresponding to the acquired hardware and target program language from the pre-stored base code DB. Here, the base code 830 may include any code for generating a program for an accelerator other than the core code operation part, for example,

Allocates memory to execute device functions (e.g., CUDA device functions), host program code to transmit data, host program code to call kernel, header code of device function, executed independently of core operation It may include device function code (eg, code related to thread ID acquisition and index initialization) and/or code for reading data necessary for core operation execution from global memory.

Although FIG. 8 shows that one accelerator program 820 is generated based on one intermediate expression 810, it is not limited thereto. For example, the code generator 230 may generate a plurality of accelerator programs 820 corresponding to each of one or more intermediate expressions 810 included in a plurality of division schedules. With this configuration, the code generator 230 can generate the program 820 for the accelerator by generating the code corresponding to the intermediate expression 810 in real time.

9 is an exemplary diagram illustrating a dataflow graph 900 of an intermediate representation according to an embodiment of the present disclosure. The dataflow graph 900 of the intermediate expression may be generated by connecting unit operation intermediate expressions (eg, data flow graphs of the unit operation intermediate expression). Each node of the graph 900 may represent a scalar operation constituting an intermediate expression, and each edge connecting the nodes may represent a dependency relationship between the scalar operations.

According to an embodiment, the above-described processor and/or code generator may search the graph 900 in reverse order traversal to generate a core operation code corresponding to an operation of each node. Here, the reverse backward traversal is a method for searching each node in a graph having an inverted tree structure, and may be a method of searching each subtree in order.

According to one embodiment, in the dataflow graph 900 of the intermediate expression, the code generator may start searching from the A node 910 at the top left. That is, the code generator may generate the code of node A 910 . Then, the code generator may generate the code of the node B 920 necessary to generate the node D 940, which is a child node of the node A 910, and then the code of the node D 940. Also, the code generator may generate the code of the F node 960, which is a child node of the D node 940. Then, by reverse postorder traversal, the code generator can generate code for C node 930, generate code for E node 950, and generate code for G node 970. After both the code of the F node 960 and the code of the G node 970 are generated, finally, the code of the H node 980 can be generated. In short, the A node (910), B node (920), D node (940), F node (960), C node (930), E node (950), G node (970) and H node (980) sequence. It is searched for, and core operation codes can be generated in that order.

in this disclosure. Each node may be searched according to an edge connecting each node (an edge corresponding to a dependency relationship of each scalar operation) by reverse postorder traversal, and a code may be generated based on the searched order. For example, the above code may be generated line-by-line, or may be generated in the form of SSA (Single Static Assignment). In addition, a new variable may be created for every line of code, and each code may be created so that variable names do not overlap.

10 is a block diagram illustrating an example of generating a final accelerator program 1060 according to an embodiment of the present disclosure. As shown, unit operation division scheduler 1020 may receive deep learning application 1010 . For example, the deep learning application 1010 can be a combination of conceptual unit operations created using a deep learning framework.

The unit operation division scheduler 1020 may generate a unit operation list 1022 included in the deep learning application 1010 . Specifically, the unit operation division scheduler 1020 extracts a plurality of unit operations included in the deep learning application 1010 by analyzing the deep learning application 1010, and a unit operation list including the plurality of extracted unit operations (1022). In addition, the unit operation division scheduler 1020 analyzes patterns of a plurality of unit operations of the deep learning application 1010 to determine the data dependency relationship of the plurality of unit operations, and based on the determined data dependency relationship of the plurality of unit operations , a plurality of division schedules corresponding to the deep learning application 1010 may be determined. In this case, the unit operation division scheduler 1020 may select one division schedule 1024 from among a plurality of division schedules.

According to an embodiment, a plurality of unit operation intermediate expressions 1032 corresponding to the unit operation list 1022 may be extracted from the intermediate expression DB 1030 based on the selected split schedule 1024 . Here, each of the plurality of unit operation intermediate expressions 1032 may represent a data flow graph for an operation of one thread block. Then, one or more intermediate expressions may be generated by connecting a plurality of unit operation intermediate expressions 1032 based on one selected division schedule 1024 .

According to one embodiment, the code generator 1040 may generate an accelerator program 1042 corresponding to each of one or more intermediate expressions generated based on the selected division schedule 1024 . Here, the intermediate expression may be composed of a graph including a plurality of nodes representing each scalar operation and an edge representing a dependency relationship between the scalar operations. For example, the code generator 1040 may generate a key operation code corresponding to an operation of each node by traversing a graph of an intermediate expression in reverse postorder traversal.

Also, the code generator 1040 may obtain hardware and target program language used for deep learning applications. For example, hardware may refer to a multi-core CPU, GPU, FPGA, etc. used for deep learning, but is not limited thereto. Also, the target program language may refer to a program language for generating a GPU program or an FPGA program using a multi-core CPU, GPU, FPGA, or the like. For example, a GPU program may include a host function for calling a kernel function (or device function), and the like. In another example, an FPGA program may include a bitstream, host functions for synthesizing and executing it into FPGA hardware, and the like. As another example, the code generator 1040 may extract a base code corresponding to the acquired hardware and target program language from a pre-stored base code DB. Then, the code generator 1040 may generate an accelerator program 1042 including the basic code and the generated core operation code.

Then, the performance measurer 1050 may determine the performance 1052 of the generated accelerator program 1042 . Also, the performance measurer 1050 may provide the determined performance 1052 of the accelerator program 1042 to the unit operation division scheduler 1020 . Based on the performance 1052 of the determined accelerator program 1042, the unit operation division scheduler 1020 may select another division schedule other than the previously selected division schedule from among a plurality of division schedules. Then, a unit operation list according to another selected splitting schedule is provided, an intermediate unit operation expression is extracted, the intermediate expression is generated, and another accelerator program may be generated.

By repeating this process, the code generator 1040 may generate a plurality of accelerator programs corresponding to the determined plurality of division schedules. In this case, the performance measurer 1050 may determine the performance of each of the plurality of accelerator programs. Then, the performance measurer 1050 may select one accelerator program 1060 from among a plurality of generated accelerator programs based on the determined performance.

In FIG. 10, the unit operation division scheduler 1020, the code generator 1040, the performance measurer 1050, etc. are classified and described according to functions, but this does not necessarily mean that they are physically separated. For example, although the code generator 1040 and the performance measurer 1050 have been separately described above, this is only for helping understanding of the invention, and one computing device may perform two or more functions. In addition, in FIG. 10, it has been described in detail that the unit operation division scheduler 1020, the code generator 1040, the performance measurer 1050, etc. are used to determine the final accelerator program 1060, but is not limited thereto, and the final accelerator In order to determine the application program 1060, an intermediate expression conversion module and the like may be further included. With this configuration, a final accelerator program with the best performance among accelerator programs according to a division schedule can be generated, and the created final accelerator program can be used to drive a deep learning application.

The above method may be provided as a computer program stored in a computer readable recording medium to be executed on a computer. The medium may continuously store programs executable by a computer or temporarily store them for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, and ROM, RAM, flash memory, etc. configured to store program instructions. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server.

The methods, acts or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or combinations thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques may include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs) ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logical blocks, modules, and circuits described in connection with this disclosure may be incorporated into a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer readable medium, such as programmable read-only memory (EPROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, or the like. It can also be implemented with stored instructions. Instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

Although the embodiments described above have been described as utilizing aspects of the presently-disclosed subject matter in one or more stand-alone computer systems, the disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. . Further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across multiple devices. These devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in relation to some embodiments in this specification, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art. Moreover, such modifications and variations are intended to fall within the scope of the claims appended hereto.

100: processor 110: deep learning application
120: program for accelerator

Claims (10)

In the method of generating a program for an accelerator for deep learning,
receiving a deep learning application;
extracting a plurality of unit operations included in the deep learning application by analyzing the deep learning application by a unit operation division scheduler;
determining a plurality of division schedules corresponding to the deep learning application by analyzing patterns of the plurality of extracted unit operations;
selecting one of the plurality of divided schedules by the unit operation division scheduler;
generating a unit operation list corresponding to the selected split schedule, wherein the unit operation list includes the extracted plurality of unit operations;
generating an intermediate expression from a unit operation list corresponding to the selected split schedule; and
Generating a program for an accelerator for the deep learning application based on the intermediate expression.
A method for generating a program for an accelerator for deep learning, including a.
delete According to claim 1,
Determining a plurality of split schedules corresponding to the deep learning application,
determining a data dependency relationship of the plurality of unit operations by analyzing patterns of the plurality of unit operations extracted; and
Determining a plurality of division schedules corresponding to the deep learning application based on the determined data dependency relationship of the plurality of unit operations
A method for generating a program for an accelerator for deep learning, including a.
According to claim 1,
Generating an intermediate expression from a unit operation list corresponding to the selected one split schedule,
extracting a plurality of unit operation intermediate expressions corresponding to the unit operation list from an intermediate expression DB based on the selected one division schedule - each of the plurality of unit operation intermediate expressions corresponds to an operation of one thread block Indicates a data flow graph for -; and
generating one or more intermediate expressions by connecting the extracted plurality of unit operation intermediate expressions based on the selected splitting schedule;
A method for generating a program for an accelerator for deep learning, including a.
According to claim 4,
Generating a program for an accelerator for the deep learning application based on the intermediate expression,
Generating a program for an accelerator corresponding to each of the one or more intermediate expressions generated based on the one selected division schedule.
A method for generating a program for an accelerator for deep learning, including a.
According to claim 1,
determining performance of the generated accelerator program;
providing the determined performance of the accelerator program to the unit operation division scheduler; and
Based on the performance of the determined accelerator program, selecting another division schedule other than the selected division schedule from among the plurality of division schedules by the unit operation division scheduler.
Further comprising a method for generating a program for an accelerator for deep learning.
According to claim 1,
generating a plurality of accelerator programs corresponding to the determined plurality of division schedules; and
Further comprising determining the performance of each of the plurality of accelerator programs,
Generating a program for an accelerator for the deep learning application based on the intermediate expression,
Based on the determined performance, selecting one accelerator program from among the generated accelerator programs.
A method for generating a program for an accelerator for deep learning, including a.
According to claim 1,
The intermediate expression is represented as a graph including a plurality of nodes representing each scalar operation and an edge representing a dependency relationship between the scalar operations,
Generating a program for an accelerator for the deep learning application based on the intermediate expression,
Generating a key operation code corresponding to the operation of each node by searching the graph of the intermediate expression in reverse postorder.
A method for generating a program for an accelerator for deep learning, including a.
According to claim 8,
Generating a program for an accelerator for the deep learning application based on the intermediate expression,
acquiring hardware and a target program language used in the deep learning application;
extracting a base code corresponding to the obtained hardware and a target program language from a pre-stored base code DB; and
Generating a program for an accelerator including the extracted basic code and the generated core operation code
A method for generating a program for an accelerator for deep learning, including a.
A computer program stored in a computer readable recording medium to execute a method for generating a program for an accelerator for deep learning according to any one of claims 1 and 3 to 9 in a computer.

Images