KR20220067289A - Gpgpu thread block scheduling method and apparatus - Google Patents

Abstract

Disclosed are a GPGPU thread block scheduling method and device. The thread block scheduling method performed by a thread block scheduler comprises: a step of identifying a remaining resource amount of computing resources for processing a plurality of different function units for each of a plurality of stream multiprocessors (SMs) constituting a GPU; a step of identifying resource usage of the computing resources according to a workload type for each of a plurality of thread blocks (TBs) to be processed through the GPU; and a step of allocating each of the TBs to one among the SMs based on the remaining resource amount of the computing resources for each of the identified SMs and the resource usage of the computing resources for each of the TBs. Accordingly, GPU task placement and multitasking can be achieved efficiently.

Description

GPGPU thread block scheduling method and apparatus {GPGPU THREAD BLOCK SCHEDULING METHOD AND APPARATUS}

The present invention relates to a method and apparatus for scheduling a GPGPU thread block, and more particularly, to a method and apparatus for scheduling a thread block based on resource usage characteristics of a workload performed on a GPU.

With the advent of the 4th industrial revolution, GPUs are being used as parallel computing devices in various fields such as deep learning, block chain, and genetic analysis. However, GPU management has focused on maximizing parallelism of individual workloads through a concise control structure rather than reflecting the diversity of workloads.

Recently, as workloads in various fields are supported to be performed simultaneously on the GPU, increasing efficiency due to multitasking rather than parallel execution of individual workloads is emerging as an important issue in GPU management.

Accordingly, there is a demand for a method for scheduling a thread block of a GPU to increase efficiency due to multitasking.

The present invention relates to a GPGPU thread block scheduling method and apparatus, and more particularly, by analyzing resource usage characteristics of a workload performed on a GPU, thread blocks corresponding to workloads having different resource usage characteristics are combined with the same stream multiprocessor. (Stream Multiprocessor, hereinafter SM) relates to a method and apparatus for deployment.

The thread block scheduling method performed by the thread block scheduler according to an embodiment of the present invention includes a plurality of different function units for each of a plurality of stream multiprocessors (SM) constituting a GPU. identifying a remaining resource amount of computing resources to process; identifying resource usage of the computing resource according to the type of workload for each of a plurality of thread blocks (hereinafter referred to as TB) to be processed through the GPU; and assigning each of the plurality of TBs to any one of the plurality of SMs based on the amount of remaining resources of the computing resources for each of the identified SMs and the resource usage of the computing resources for each of the plurality of TBs. It may include the step of allocating to.

The computing resource may include at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit.

In the allocating step, when two or more TBs to be allocated to a specific SM use different computing resources, the allocation may be performed to the same SM within a limit that does not exceed the remaining resource amount for the computing resources of the specific SM.

In the allocating step, each of the plurality of TBs may be sequentially allocated one by one to the plurality of SMs according to a round-robin method.

A thread block scheduler according to an embodiment of the present invention includes a processor, wherein the processor includes a plurality of different function units for each of a plurality of stream multiprocessors (SM) constituting the GPU. Identifies the amount of remaining resources of the computing resource for processing the Allocating each of the plurality of TBs to any one SM of the plurality of SMs based on the amount of remaining resources of the computing resources for each of the identified SMs and the resource usage of the computing resources for each of the plurality of TBs can do.

The computing resource may include at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit.

When two or more TBs to be allocated to a specific SM use different computing resources, the processor may allocate to the same SM within a limit that does not exceed the remaining resource amount for the computing resources of the specific SM.

The processor may sequentially allocate each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method.

The thread block scheduling method performed by the thread block scheduler according to an embodiment of the present invention identifies the remaining resource amounts of computing resources and memory resources for each of a plurality of stream multiprocessors (SM) constituting a GPU. step; identifying resource usage of the GPU according to a workload type for each of a plurality of thread blocks (hereinafter referred to as TB) to be processed through the GPU; and allocating each of the plurality of TBs to any one of the plurality of SMs based on the amount of remaining resources for each of the identified SMs and the resource usage of the GPU for each of the plurality of TBs. Including, wherein the computing resource is composed of devices for processing a plurality of different function units (Function Unit), the memory resource may be composed of a register file, a shared memory and a unified cache.

The computing resource may include at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit.

In the allocating step, if computing resources and memory resources for two or more TBs to be allocated to a specific SM do not exceed the remaining resource amount of the specific SM, allocating the TBs to the specific SM, and the remaining resource amount of the specific SM In case of exceeding , a specific SM may be skipped.

In the allocating step, each of the plurality of TBs may be sequentially allocated one by one to the plurality of SMs according to a round-robin method.

A thread block scheduler according to an embodiment of the present invention includes a processor, wherein the processor identifies the remaining resource amounts of computing resources and memory resources for each of a plurality of stream multiprocessors (SM) constituting the GPU. and identify the resource usage of the GPU required based on the workload type for each of a plurality of thread blocks (hereinafter referred to as TBs) to be processed through the GPU, and the remaining amount of each of the identified plurality of SMs Allocating each of the plurality of TBs to any one of the SMs based on the amount of resources and the GPU resource usage of each of the plurality of TBs, and the computing resources are different from each other a plurality of function units (Function Unit) It consists of devices for processing, and the memory resource may consist of a register file, a shared memory, and a unified cache.

The computing resource may include at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit.

If the computing resources and memory resources for two or more TBs to be allocated to the specific SM do not exceed the remaining resource amount of the specific SM, the processor allocates the TBs to the specific SM and exceeds the remaining resource amount of the specific SM In this case, the specific SM may be skipped.

The processor may sequentially allocate each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method.

The present invention relates to a GPGPU thread block scheduling method and apparatus, and more particularly, by analyzing resource usage characteristics of a workload performed on a GPU, thread blocks corresponding to workloads having different resource usage characteristics are arranged in the same SM By doing so, GPU work placement and multitasking can be achieved efficiently.

1 is a diagram illustrating a conceptual diagram of thread block scheduling according to an embodiment of the present invention.
2 is a diagram illustrating resources of an SM required to process a thread block according to an embodiment of the present invention.
3 is a diagram illustrating a first method in which a TBS allocates a specific TB to an SM according to an embodiment of the present invention.
4 is a diagram illustrating a second method in which a TBS allocates a specific TB to an SM according to an embodiment of the present invention.
5 is a flowchart illustrating a thread block scheduling method of a GPGPU according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a conceptual diagram of thread block scheduling according to an embodiment of the present invention.

The computing hardware of the GPU is a set of computing processors (mainly Stream Multiprocessor, hereinafter SM) composed of hundreds or thousands of computational cores. The number of such SMs may vary according to the type of GPU device. Usually, one SM can perform tens to hundreds of identical computing commands at the same time in hardware by changing only the processing data.

The programming method mainly used for GPU is a hardware parallel execution method of a single-threaded program based on one core. However, since the computing problems to be solved through the GPU can consist of hundreds of thousands to millions of threads, it is impossible to perform them simultaneously on the GPU in terms of hardware.

To solve this problem, the GPU provides a method of grouping into a task unit of a certain size according to the problem to be solved from hundreds of thousands to millions of threads and performing it on SM. This unit of work is called a thread block (hereinafter referred to as TB), and a manager that allocates TBs to a plurality of SMs in the GPU device is called a thread block scheduler (hereinafter, TBS).

Referring to FIG. 1 , the TBS 100 may perform a role of receiving TBs included in a plurality of different sub-kernels and allocating them to SM, which is hardware in the GPU. Here, the sub-kernels are codes executed on the GPU and may be understood as programs in terms of the CPU concept. The sub-kernels may include TBs of different sizes depending on the type of kernel, and the TBs included in each sub-kernel may have the same size. That is, the sub-kernel A (Kernel A) and the kernel B (Kernel B) may include TBs of different sizes, and the TBs included in each of the sub-kernel A and the sub-kernel B may have the same size.

The TB may consist of 1 thread to usually 1024 threads, and the SM may concurrently perform threads in the TB allocated through the TBS 100 . A warp represents a unit of threads that can be physically simultaneously executed in the hardware of the GPU, and usually one warp can consist of 32 threads. For example, if it is assumed that a TB is composed of 64 threads, the corresponding TB may be composed of two warps. And, when the corresponding TB is allocated to the SM through the TBS 100, it can be simultaneously performed in 32 threads, ie, warp units through the allocated SM.

If a TB consists of one thread, the TB can consist of one warp. Since a TB with one thread is not used for purposes other than testing, the number of threads in a TB is usually a multiple of 32. have.

As such, the throughput of a task (thread) performed in the SM, which is hardware in the GPU, may be greatly affected by the scheduling policy performed by the TBS 100 . The TBS 100 may sequentially allocate TBs included in a plurality of different sub-kernels to a plurality of SMs one by one in a round-robin manner.

At this time, the TBS 100 provided in the present invention analyzes the resource usage characteristics of the workload performed on the GPU and arranges thread blocks corresponding to the workloads having different resource usage characteristics in the same SM, thereby providing GPU work arrangement and multi-tasking. It can provide a way to achieve the task efficiently.

2 is a diagram illustrating resources of an SM required to process a thread block according to an embodiment of the present invention.

When a TB is allocated to the SM constituting the GPU, a resource of a certain size may be required to process the allocated TB. In this case, the required resource may include at least one of the number of threads, shared memory, and the number of registers.

Since the TBs included in the same kernel may have the same size, the required amount of SM resource may be the same, and this amount of SM resource may be determined when the TBs are compiled.

When a specific TB is allocated to any one of a plurality of SMs, the TBS 100 may first identify a residual resource amount for each of the plurality of SMs. Thereafter, the TBS 100 compares the remaining resource amount of the first SM with the SM resource usage for the specific TB to be allocated according to a predetermined allocation order. When the SM resource usage of the specific TB is smaller than the remaining resource amount of the first SM, A TB may be allocated to the first SM.

On the other hand, when the SM resource usage of a specific TB is greater than the remaining resource amount of the first SM, the TBS 100 skips the first SM, and thereafter, the remaining resource amount of the second SM corresponding to the allocation sequence and the specific TB SM resource usage may be compared, and allocation may be performed according to the comparison result.

At this time, the TBS 100 is the first SM when the remaining resource amount for at least one of the number of threads, the shared memory, and the number of registers constituting the resource of the first SM is smaller than the SM resource usage of the specific TB to be allocated. After skipping, it may be determined whether a specific TB is allocated to the second SM corresponding to the allocation order.

3 is a diagram illustrating a first method in which a TBS allocates a specific TB to an SM according to an embodiment of the present invention.

Since the SM processing the TBs has limited resources, it may be important to properly place the TBs within the limited resources through the TBS 100 . TBs processed through the SM of the GPU may have different SM resources depending on the type of workload. For example, referring to FIG. 3 , it can be seen that a TB corresponding to a hotspot mainly uses a computing resource among SM resources, and a TB corresponding to a stream cluster mainly uses a memory resource among SM resources.

Accordingly, the TBS 100 determines whether the TBs to be allocated are computing resource intensive or memory resource intensive based on the type of workload, and jointly allocates TBs using different types of resources intensively to one SM. By doing so, the resource utilization rate of SM can be maximized.

In order to determine whether TBs to be allocated are computing resource intensive or memory resource intensive, a pre-use test may be performed after kernel build. Thereafter, the test result value may be reflected in a plurality of TBs included in the kernel.

4 is a diagram illustrating a second method in which a TBS allocates a specific TB to an SM according to an embodiment of the present invention.

In FIG. 3, it is determined whether the TB to be allocated is computing resource intensive or memory resource intensive, and according to the determination result, TBs using different types of resources intensively are jointly allocated to one SM to determine the resource usage rate of the SM. A way to maximize is provided.

Meanwhile, computing resource-intensive workloads may have different SM resources required according to the type of a function unit. At this time, the computing resource is a single-precision (Single Floating Point, SFP) arithmetic unit, a double-precision (Double Floating Point, DFP) arithmetic unit, a control-flow (CF) unit, a load/store ( It may include at least one of a Load/Store, LSDT) device and a Special Function Unit (SFU) arithmetic unit.

As an example, referring to FIG. 4 , the TB corresponding to Back Propagation mainly uses a double-precision arithmetic unit among the SM's computing resources, and the TB corresponding to the Nbody mainly uses a single-precision arithmetic unit among the SM's computing resources. Able to know.

Accordingly, the TBS 100 determines which resource among a plurality of computing resources is intensively used for the TBs to be allocated based on the workload type, and TBs using different types of computing resources intensively are assigned to one SM. By jointly allocating, the resource utilization rate of SM can be maximized.

In order to determine which of the computing resources the TBs to be allocated intensively use in this way, a pre-use test may be performed after the kernel is built. Thereafter, the test result value may be reflected in a plurality of TBs included in the kernel.

5 is a flowchart illustrating a thread block scheduling method of a GPGPU according to an embodiment of the present invention.

In step 510 , the TBS 100 may identify the remaining amount of computing resources and memory resources for each of the plurality of SMs constituting the GPU. In this case, the computing resource is a device for processing a plurality of different functional units, and may include at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit. In addition, the memory resource may include at least one of a register file, a shared memory, and an integrated cache.

In step 520 , the TBS 100 may identify the resource usage of the GPU according to the workload type for each of the plurality of TBs to be processed through the GPU.

In step 530 , the TBS 100 assigns each of the plurality of TBs to any one of the plurality of SMs based on the amount of remaining resources for each of the identified SMs and the resource usage of the GPU for each of the plurality of TBs. can be assigned to the SM of

At this time, when the computing resources and memory resources for two or more TBs to be allocated to the specific SM do not exceed the remaining resource amount of the specific SM, the TBS 100 allocates the TBs to the specific SM and the remaining resource amount of the specific SM. If it exceeds, the specific SM may be skipped, and it may be determined whether to allocate the SM corresponding to the subsequent allocation order.

Meanwhile, the method according to the present invention is written as a program that can be executed on a computer and can be implemented in various recording media such as magnetic storage media, optical reading media, and digital storage media.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or combinations thereof. Implementations may be implemented for processing by, or controlling the operation of, a data processing device, eg, a programmable processor, computer, or number of computers, a computer program product, ie an information carrier, eg, a machine readable storage It may be embodied as a computer program tangibly embodied in an apparatus (computer readable medium) or a radio signal. A computer program, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, as a standalone program or in a module, component, subroutine, or computing environment. It can be deployed in any form, including as other units suitable for use in A computer program may be deployed to be processed on one computer or multiple computers at one site or distributed across multiple sites and interconnected by a communications network.

Processors suitable for processing a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. In general, a processor will receive instructions and data from either read-only memory or random access memory or both. Elements of a computer may include at least one processor that executes instructions and one or more memory devices that store instructions and data. In general, a computer may include, receive data from, transmit data to, or both, one or more mass storage devices for storing data, for example magnetic, magneto-optical disks, or optical disks. may be combined to become Information carriers suitable for embodying computer program instructions and data are, for example, semiconductor memory devices, for example, magnetic media such as hard disks, floppy disks and magnetic tapes, Compact Disk Read Only Memory (CD-ROM). ), optical recording media such as DVD (Digital Video Disk), magneto-optical media such as optical disk, ROM (Read Only Memory), RAM (RAM) , Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and the like. Processors and memories may be supplemented by, or included in, special purpose logic circuitry.

In addition, the computer-readable medium may be any available medium that can be accessed by a computer, and may include both computer storage media and transmission media.

While this specification contains numerous specific implementation details, they should not be construed as limitations on the scope of any invention or claim, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. should be understood Certain features that are described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features operate in a particular combination and may be initially depicted as claimed as such, one or more features from a claimed combination may in some cases be excluded from the combination, the claimed combination being a sub-combination. or a variant of a sub-combination.

Likewise, although acts are depicted in the drawings in a particular order, it should not be construed that all acts shown must be performed or that such acts must be performed in the specific order or sequential order shown to obtain desirable results. In certain cases, multitasking and parallel processing may be advantageous. Further, the separation of the various device components of the above-described embodiments should not be construed as requiring such separation in all embodiments, and the program components and devices described may generally be integrated together into a single software product or packaged into multiple software products. You have to understand that you can.

On the other hand, the embodiments of the present invention disclosed in the present specification and drawings are merely presented as specific examples to aid understanding, and are not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

100: Thread Block Scheduler (TBS)

Claims (16)

A thread block scheduling method performed by a thread block scheduler, comprising:
identifying a residual resource amount of a computing resource for processing a plurality of different function units for each of a plurality of stream multiprocessors (hereinafter referred to as SM) constituting the GPU;
identifying resource usage of the computing resource according to the type of workload for each of a plurality of thread blocks (hereinafter referred to as TB) to be processed through the GPU; and
Each of the plurality of TBs to any one SM of the plurality of SMs based on the amount of remaining resources of the computing resources for each of the identified SMs and the resource usage of the computing resources for each of the plurality of TBs Steps to assign
A thread block scheduling method comprising: According to claim 1,
The computing resource is
A thread block scheduling method comprising at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit. According to claim 1,
The allocating step is
When two or more TBs to be allocated to a specific SM use different computing resources, a thread block scheduling method for allocating to the same SM within a limit that does not exceed the amount of remaining resources for the computing resources of the specific SM. According to claim 1,
The allocating step is
A thread block scheduling method for sequentially allocating each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method. In the thread block scheduler,
including a processor;
The processor is
For each of the plurality of stream multiprocessors (hereinafter referred to as SM) constituting the GPU, the remaining resource amount of the computing resource for processing a plurality of different function units is identified, and the plurality of streams to be processed through the GPU are identified. Identifies the resource usage of the computing resource according to the type of workload for each of the Thread Blocks (TBs) of A thread block scheduler for allocating each of the plurality of TBs to any one of the plurality of SMs based on the resource usage of the computing resource for each. 6. The method of claim 5,
The computing resource is
A thread block scheduler comprising at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit. 6. The method of claim 5,
The processor is
When two or more TBs to be allocated to a specific SM use different computing resources, the thread block scheduler allocates to the same SM within a limit that does not exceed the amount of remaining resources for the computing resources of the specific SM. 6. The method of claim 5,
The processor is
A thread block scheduling method for sequentially allocating each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method. A thread block scheduling method performed by a thread block scheduler, comprising:
identifying the remaining resource amounts of computing resources and memory resources for each of a plurality of stream multiprocessors constituting the GPU (Stream Multiprocessor, hereinafter SM);
identifying resource usage of the GPU according to the type of workload for each of a plurality of thread blocks (hereinafter referred to as TB) to be processed through the GPU; and
Allocating each of the plurality of TBs to any one of the plurality of SMs based on the amount of remaining resources for each of the identified SMs and the resource usage of the GPU for each of the plurality of TBs
including,
The computing resource is
It consists of devices for processing a plurality of different function units,
The memory resource is
A method for scheduling thread blocks that consists of a register file, shared memory, and a unified cache. 10. The method of claim 9,
The computing resource is
A thread block scheduling method comprising at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit. 10. The method of claim 9,
The allocating step is
If the computing resources and memory resources for two or more TBs to be allocated to a specific SM do not exceed the remaining resource amount of the specific SM, allocating the TBs to the specific SM and exceeding the remaining resource amount of the specific SM, A thread block scheduling method that skips a specific SM. 10. The method of claim 9,
The allocating step is
A thread block scheduling method for sequentially allocating each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method. In the thread block scheduler,
including a processor;
The processor is
A plurality of stream multiprocessors (hereinafter, SM) constituting the GPU are identified, and the remaining resource amounts of computing resources and memory resources are identified, and a plurality of thread blocks to be processed through the GPU (Thread Blocks, hereinafter referred to as TBs) are selected. Identifies the resource usage of the GPU required based on the workload type for each, and based on the residual resource amount of each of the identified plurality of SMs and the GPU resource usage of each of the plurality of TBs, each of the plurality of TBs is assigned to any one of the plurality of SMs,
The computing resource is
It consists of devices for processing a plurality of different function units,
The memory resource is
A thread block scheduler consisting of a register file, shared memory, and a unified cache. 14. The method of claim 13,
The computing resource is
A thread block scheduler comprising at least one of a single-precision arithmetic unit, a double-precision arithmetic unit, a control flow unit, a load/store unit, and a special function arithmetic unit. 14. The method of claim 13,
The processor is
If the computing resources and memory resources for two or more TBs to be allocated to a specific SM do not exceed the remaining resource amount of the specific SM, allocating the TBs to the specific SM and exceeding the remaining resource amount of the specific SM, A thread block scheduler that skips that specific SM. 14. The method of claim 13,
The processor is
A thread block scheduler that sequentially allocates each of the plurality of TBs one by one to the plurality of SMs according to a round-robin method.

Images