KR102505279B1 - Method for optimizing parallel matrix multiplication in a system supporting multiple CPU and multiple GPU - Google Patents

Abstract

A multi-process computing method and computing system are provided. A computing method for multi-processes in a computing system of the present disclosure estimates a block size to be distributed to a plurality of GPUs connected to a CPU, and performs a matrix multiplication operation having the estimated block size in the memory of each of the plurality of GPUs. carry out The estimated block size is a value estimated through machine learning based on the number of GPUs connected to the CPU and the size of a square matrix obtained through a matrix multiplication operation of the plurality of GPUs.

Description

Operation method in a computing environment supporting multiple CPUs and multiple GPUs {Method for optimizing parallel matrix multiplication in a system supporting multiple CPU and multiple GPU}

The present disclosure relates to computer processing technology, and more particularly, to a method and system for providing optimal numerical calculation (parallel matrix multiplication) in a computing environment composed of a plurality of CPUs (Central Processing Units) and a plurality of GPUs (Graphics Processing Units). It is about.

As the demand for processing large amounts of data such as big data increases, the demand for a computing environment using multiple CPUs and multiple GPUs is increasing. Therefore, there is a need for a calculation method that speeds up numerical calculation in a system composed of multiple CPUs and multiple GPUs. GPUs use many cores in parallel to perform calculations on large amounts of data simultaneously, greatly speeding up calculations.

Conventionally, when data is exchanged between a plurality of GPUs in a computing environment including a plurality of GPUs and a plurality of CPUs, operation data generated by performing an operation on each GPU is a QPI (QuickPath Interconnect) connecting the plurality of CPUs. It must pass through a communication path interconnecting a plurality of CPUs, such as a channel.

Also, when processing large amounts of data, large scale matrix multiplication is used. Conventionally, various techniques for block decomposition of a matrix have been proposed in a system in which a multi-processor is implemented. On the other hand, in systems using multi-GPUs, technology development for optimal block separation of matrices is in an early stage. Therefore, a method for finding an optimal block size capable of improving matrix operation speed in a multi-GPU system is required.

An object of the present disclosure is to provide a method and system for speeding up parallel operation in a plurality of GPUs connected to a CPU by estimating an optimal block size through machine learning.

In order to achieve the above object, according to an embodiment of the present disclosure, a multi-process computing system includes a first CPU (Core Processing Unit) and a first memory through a first connector. A plurality of first GPUs (Graphical Processing Units, GPUs) connected to the second, in order to achieve the above-described object, according to an embodiment of the present disclosure, a multi-process computing system, CPU (Central Processing Unit), a plurality of GPUs (Graphics Processing Units) connected to the CPU through a connector, and a memory included in each of the plurality of GPUs, wherein each of the plurality of GPUs includes a block designated in the memory of each GPU. A matrix multiplication operation having a size is performed, and the designated block size is based on the number of a plurality of GPUs connected to the CPU and the size of a square matrix obtained through the matrix multiplication operation of the plurality of GPUs It is an estimated value using machine learning.

In addition, in order to achieve the above object, a computing method for multi-processes according to an embodiment of the present disclosure includes the steps of estimating a block size to be distributed to a plurality of GPUs connected to a CPU through a connector and the and performing a matrix multiplication operation having the estimated block size in a memory of each of a plurality of GPUs, wherein the estimated block size is determined by the number of GPUs connected to the CPU and the matrix multiplication of the plurality of GPUs. It is an estimated value using machine learning based on the size of the square matrix obtained through the operation.

Data processing methods according to embodiments of the present disclosure may provide high performance computing (HPC) by performing high-speed parallel calculations through a plurality of GPUs in a computing system including a plurality of GPUs connected to a CPU. .

1 is a simplified block diagram of a computing system, according to one embodiment of the present disclosure;
2 is a detailed block diagram of a computing system, according to an embodiment of the present disclosure;
3 is a simplified block diagram of a computing system according to an embodiment of the present disclosure;
2 is a detailed block diagram of a computing system, according to an embodiment of the present disclosure;
3 is a flowchart illustrating an operation method in a plurality of GPUs according to an embodiment of the present disclosure;
4 is a diagram illustrating an operation method in a plurality of GPUs according to embodiments of the present disclosure;
5 is a flowchart illustrating a method of estimating an optimal block size through machine learning according to an embodiment of the present disclosure;
6 is a graph showing execution time according to block size, according to an embodiment of the present disclosure, and
7 is a table showing results of estimating an optimal block size according to an embodiment of the present disclosure.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary according to the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like.

In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

Embodiments of the present disclosure may apply various transformations and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of technology disclosed. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the subject matter, the detailed description will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. Terms are only used to distinguish one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "consist of" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented in hardware or software or a combination of hardware and software. In addition, a plurality of "modules" or a plurality of "units" are integrated into at least one module and implemented by at least one processor (not shown), except for "modules" or "units" that need to be implemented with specific hardware. It can be.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly describe the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a simplified block diagram of a computing system, according to one embodiment of the present disclosure. Referring to FIG. 1 , a computing system 10 includes a plurality of CPUs (Central Processing Units) 110 and 210 , each CPU 110 and 210 and each memory 130 and 230 as a plurality of GPUs (Graphic Processing Units). A plurality of connectors 120 and 220 connected to (140, 150, 240, and 250) may be included.

System 10 may be, for example, a supercomputer comprising multiple CPUs and multiple GPUs. Additionally, system 10 may be a computing system including one CPU and multiple GPUs. System 10 may be implemented in a single electronic device such as a computer.

Each of the CPUs 110 and 210 may be a chipset including a system memory. Each of the GPUs 140, 150, 240, and 250 may be a special purpose processor for processing graphic data such as video games. For example, the GPU may be an NVIDA ^™ GPU, but is not limited thereto.

The connectors 120 and 220 include a plurality of GPUs 140, 150, 240, and 250 connected to the connectors 120 and 220 through a bus, which is a plurality of network interfaces, and memory connected to the respective connectors 120 and 220 ( 130, 230) and CPU (110, 210) to be able to communicate.

For example, the connectors 120 and 220 may be a root complex in a peripheral component interconnect express (PCIe ^TM ) system that is a bridged host interface. In the present disclosure, for convenience of explanation, a computing system implemented as a PCIe ^TM system is described, but is not limited thereto. The term “root complex” used in this disclosure may mean “connector”.

According to an embodiment of the present disclosure, GPU1 140 and GPU2 150, which are a plurality of first graphics processing units (GPUs), connect a first central processing unit (CPU) through a first connector 120. ) 110 and the first memory 130. The plurality of second GPUs, GPU3 240 and GPU 250 , may be connected to the second CPU 210 and the second memory 230 through a second connector 220 . In addition, the first CPU 110 and the second CPU 210 may communicate through a communication path interconnecting the respective CPUs 110 and 210 .

According to an embodiment of the present disclosure, each of the plurality of first GPUs 140 and 150 may generate first calculation data by performing a first calculation based on address information stored in each other's memories. Also, each of the plurality of second GPUs 240 and 250 may perform a second operation based on address information stored in each other's memory to generate second operation data. And, the first operation data and the second operation data can be exchanged through a communication path.

For example, each of the plurality of first GPUs 140 and 150, which are a plurality of first GPUs commonly connected to the first connector 120, performs calculations and generates first calculation data, and is commonly connected to the second connector 220. The GPUs 240 and 250, which are a plurality of connected second GPUs, may each perform an operation to generate second operation data.

For another example, GPU1 140 and GPU2 150 connected to the first connector 120 may share one switch shown in FIG. 2 . Also, the GPU1 140 and the GPU2 150 may be connected to different switches and connected to one first connector 120 .

When GPU1 140 and GPU2 150 are connected to different switches and connected to the first connector 120, GPU1 140 and GPU2 150 are connected to each other through a bus such as NVIDIA Lane ^TM (NVIDIA Lane ^TM ) may be In this case, the GPU1 140 and the GPU2 150 may perform operations by directly requesting address information from each other through a connected bus without going through the first connector 120 .

Since the operating methods of the GPU3 240 and the GPU4 250 connected to the second connector 220 correspond to the operating methods of the GPU1 140 and the GPU2 150 connected to the first connector 120 described above, detailed descriptions are omitted. do.

Although FIG. 1 shows that two GPUs 140, 150, 240, and 250 are connected to each connector 120 and 220, this is only an example, and each connector 120 and 220 may be connected to two or more GPUs. can In addition, although each of the plurality of CPUs 110 and 210 in the computing system 10 may include one or more connectors 120 and 220, the computing system 10 includes a connector among the plurality of CPUs 110 and 210. It may also include CPUs that do not.

Each of the plurality of first GPUs 140 and 150 and the plurality of second GPUs 240 and 250 may include a memory. Also, the operation performed in each GPU may be a parallel matrix multiplication operation performed to access another GPU based on an address stored in a memory of each GPU. The GPU's parallel matrix multiplication operation can be performed in the memory of each GPU.

In one embodiment of the present disclosure, the target address of GPU2 150, which is at least one GPU among GPUs 140, 150, 240, and 250, is of GPU3 240 connected to a connector 220 different from that of GPU2 150. In the case of memory, operation data exchange between GPU2 (150) and GPU3 (240) connected to GPU2 (150) and another connector (220) is via a communication path interconnecting CPU1 (110) and CPU2 (210). can be performed

For example, the communication path may be Hyper Transport (HT) or QPI (QuickPath Interconnect), but is not limited thereto.

Also, the plurality of first GPUs 140 and 150 and the plurality of second GPUs 240 and 250 may be connected to the respective connectors 120 and 220 through switches. The number of switches connected to the first connector 120 and the second connector 220 may be plural. A switch connecting the connectors 120 and 220 and the plurality of GPUs 140, 150, 240 and 250 will be described with reference to FIG. 2 .

At least one of the plurality of GPUs 140 , 150 , 240 , and 250 connected to each of the CPUs 110 and 120 may be a master, and the remaining GPUs may be slaves. For example, when four GPUs are connected to the first CPU 110, one of the four GPUs may be a master and the other three may be slaves. The master GPU may build a database of the size of the input matrix of each GPU, the number of GPUs connected to the CPU, and the block size for matrix multiplication operation in the plurality of GPUs in the memory of the master GPU or the host memory of the CPU.

According to an embodiment of the present disclosure, the master GPU may estimate an optimal block size for performing a shortest matrix multiplication operation by a plurality of GPUs using machine learning based on data stored in a database. A method of estimating the optimal block size will be described in detail with reference to FIGS. 5 to 7 .

2 is a detailed block diagram of a computing system, according to one embodiment of the present disclosure. Referring to FIG. 2 , the computing system 10 may include a plurality of CPUs such as a first CPU 110 and a second CPU 210 . Each of the CPUs 110 and 210 may be connected to the first root complex 120 and the second root complex 220 through a PCIe ^TM bus, respectively. The first root complex 120 and the second root complex 220 include ^a plurality of PCIe ^TM device downstream ports 270 and can access the respective memories 130 and 230 .

Each of the root complexes 120 and 220 may communicate with the PCIe ^TM switches 160 and 260 connected to the plurality of GPUs through the PCIe ^TM bus. One of the PCIe ^TM device downstream ports 270 of the root complex 120, 220 communicates with the PCIe ^TM device upstream port 280 of the PCIe ^TM switches 160, 260 via the PCIe ^{TM bus} . can communicate

The PCIe ^TM switches 106 and 260 include a plurality of PCIe ^TM device downstream ports 270, and the PCIe ^TM device downstream port 270 is connected to a PCIe ^TM device upstream port 280 of one GPU 140 and 150. ) and the PCIe ^TM bus. Each of the GPUs 140, 150, 240, and 250 may include GPU memories 140-1, 150-1, 240-1, and 250-1, respectively. Each GPU (140, 150, 240, 250) can access the memory (140-1, 150-1, 240-1, 250-1) of each GPU through the PCIe ^TM bus. The memories 140-1, 150-1, 240-1, and 250-1 of each GPU may perform a number of parallel matrix multiplication operations.

For example, the plurality of CPUs 110 and 210 constituting the computing system 10 may be interconnected through QuickPath Interconnect (QPI).

However, the configuration of the computing system 10 shown in FIG. 2 is only one embodiment for explaining the present disclosure, is not limited thereto, and may be implemented in various ways.

3 is a flowchart illustrating an operation method in a plurality of GPUs according to an embodiment of the present disclosure.

In step S310, a parallel matrix multiplication operation is performed on each of a plurality of GPUs connected to each root complex constituting the computing system 10. The parallel matrix multiplication operation is accessed to the memory of another GPU based on the address stored in the memory of each GPU, and may be performed in each memory of the GPU.

For example, a parallel matrix multiplication operation performed on a GPU may be a square matrix of C=AxB. In this case, A may be an input matrix (NxN) of the first GPU, B may be an input matrix (NxN) of the second GPU, and C may be a square matrix (NxN) according to the parallel matrix multiplication operation of the first GPU and the second GPU. . Specifically, C(i) is a row vector of an NxN square matrix and may be the following function.

Here, |B| is a block in the row of the matrix belonging to S _i , S _i is the ith GPU, A(i) is |B|/S, and S is the number of GPUs included in the CPU. The number, B(i, j), may be |B|/S, and |B|/S may be a subblock of B(i).

Conventionally, when each GPU performs a parallel matrix multiplication operation in the computing system 10 including a plurality of CPUs, each CPU including a plurality of GPUs, the CPU through a communication path such as QPI for every process during the operation. You have to swap B(j) for another CPU. Therefore, there is a problem that the speed of calculation is lowered.

According to an embodiment of the present disclosure, a matrix multiplication operation of a plurality of GPUs may be performed in a memory of each GPU. In addition, a matrix multiplication operation of a plurality of GPUs is first performed among GPUs connected to a common root complex. Therefore, the GPU calculation method according to an embodiment of the present disclosure can improve the calculation speed because it does not have to pass through a communication path such as QPI every time the GPU performs calculation as in the prior art.

In step S330 , operation data is exchanged between a plurality of GPUs connected to different root complexes among a plurality of GPUs constituting the computing system 10 . Parallel matrix multiplication operations on GPUs can be accessed on other GPUs based on addresses stored in the memory of each GPU. Accordingly, the target address of at least one GPU among a plurality of GPUs connected to a common root complex may be one of a plurality of GPUs connected to another root complex.

For example, when a plurality of root complexes are connected to a common CPU, the target address of at least one of the plurality of GPUs connected to the common root complex is one of the plurality of GPUs connected to another root complex connected to the common CPU. can At this time, each GPU can perform calculation data exchange generated as a result of calculation within a common CPU without passing through a communication path such as QPI. Therefore, calculation speed can be improved compared to the prior art.

In step S350, GPU operation data is exchanged between a plurality of CPUs constituting the computing system 10. As described above in step S330, the target address of at least one GPU among a plurality of GPUs connected to a common root complex may be one of a plurality of GPUs connected to another root complex.

For example, a target address of a first GPU connected to a first root complex of a first CPU may be in a memory of a second GPU connected to a second root complex of a second CPU. At this time, the first GPU may exchange first operation data obtained from a plurality of first GPUs of the first CPU and second operation data obtained from a plurality of second GPUs of the second CPU through a communication path such as QPI. can

4 is a diagram illustrating an operation method in a plurality of GPUs according to embodiments of the present disclosure.

Referring to Figure 4, a thin line (thin line) 410 is a plurality of GPUs (GPU1 and GPU2, GPU3 and GPU4, GPU5 and GPU6, GPU7 and GPU8) connected through a common root complex and a common switch. shows A plurality of GPUs (GPU1 and GPU2, GPU3 and GPU4, GPU5 and GPU6, GPU7 and GPU8) may generate operation data by performing a parallel matrix multiplication operation based on address information stored in the memory of each GPU. Each GPU can exchange generated operation data with each other.

For example, GPU1 and GPU2 may be connected through a bus such as a PCIe ^TM bus, and GPU1 and GPU2 may perform operations by directly accessing respective memories. Each of GPU3 and GPU4, GPU5 and GPU6, and GPU7 and GPU8 can perform calculations in the same way as GPU1 and GPU2.

Dot line 420 shows data exchange on multiple GPUs (GPU1 and GPU4, GPU2 and GPU3, GPU5 and GPU8, GPU6 and GPU7) connected to a common root complex through different switches.

For example, GPU1 may exchange data with GPU4 via a first root complex through a PCIe ^TM bus to a first switch. Each of GPU2 and GPU3, GPU5 and GPU8, and GPU6 and GPU7 can exchange data in the same way as GPU1 and GPU3.

A thick line 430 shows data exchange between a CPU and multiple GPUs that do not share a root complex.

For example, using a communication path such as a QPI interconnecting a first CPU and a second CPU through a PCIe ^TM bus connecting each device (CPU, root complex, switch, memory, GPUs) of the computing system 10 to perform data exchange with GPU8. GPU4 can exchange data with GPU5 using the QPI communication path. At this time, when GPU1 and GPU8, GPU4 and GPU5 exchange data with each other, each GPU can pass through a switch, root complex, and CPU connected through a PCIe ^TM bus.

Each root complex may receive an access request from each GPU to a memory connected to each root complex. The root complex can receive the results of data exchange between multiple GPUs on behalf of the CPU and store the results in memory attached to (contained in) the root complex. The root complex may transmit data exchange results of the plurality of GPUs to the CPU, and the CPU may store the received data in a host memory included in the CPU.

That is, in the prior art, GPU data could be exchanged using QPI as much as the number of GPUs included in the computing system 10 (the embodiment of FIG. 4: 8), but the method according to the embodiment of the present disclosure QPI can be used as many as the number of CPUs included in ) (the embodiment of FIG. 4: 2). Therefore, the method for exchanging data between a plurality of GPUs according to the above-described embodiments of the present disclosure can significantly reduce the number of times of using QPI compared to the conventional method, thereby improving the calculation speed of graphic processing.

5 is a flowchart illustrating a method of estimating an optimal block size through machine learning, according to an embodiment of the present disclosure.

For example, a plurality of GPUs may perform parallel matrix multiplication using a programming model such as NVIDIA CUDA ^TM . However, this is only one embodiment for explaining the present disclosure and is not limited thereto.

In step S510, the computing system 10 may build a database for estimating an optimal block size in parallel matrix multiplication of a plurality of GPUs connected to one CPU.

The computing system 10 may split an input matrix of a square matrix by a matrix multiplication operation of a plurality of GPUs into a plurality of blocks. The computing system 10 may distribute a plurality of separated blocks to the master GPU and the slave GPU, respectively. Therefore, when the optimal block size to be distributed to each GPU is extracted from the input square matrix, the operation speed of the plurality of GPUs can be optimized.

The computing system 10 may determine one of the plurality of GPUs included in one CPU as a master and the remaining GPUs as slaves. The computing system 10 may store the number (n) of GPUs included in one CPU, the size (s) of an input matrix of each GPU, and the block size (b) of a square matrix in a database. The database may be the memory of the master GPU or the big data storage of the computing system 10. In step S520, the computing system 10 performs deep learning based on the database built in step S510. An optimal block size can be estimated using machine learning. In block matrix decomposition, the block size determines the execution time of the GPU when performing a parallel matrix multiplication operation through multiple GPUs ( play an important role in run time. Accordingly, it is possible to reduce the data processing time through the GPU by estimating the optimal block size.

6 is a graph illustrating execution time according to block size, according to an embodiment of the present disclosure.

As described in FIG. 3, for example, it may be a square matrix C=AxB according to matrix multiplication operation in two GPUs. In this case, A may be an NxN matrix of the first GPU, B may be an NxN matrix of the second GPU, and C may be an NxN square matrix that is a parallel multiplication matrix of the A matrix and the B matrix.

6 is a graph showing a matrix multiplication operation speed according to a block size using four GPUs for random matrix multiplication of a 10,000 x 10,000 (AxB) square matrix.

Referring to FIG. 6, when the number of blocks is small (# of Block), it means that the block size of the square matrix is decomposed relatively large, and it can mean that a lot of data is allocated in one block. In this case, since the amount of calculation data in the plurality of GPUs processed in one block increases, the input matrix calculation processing time may decrease. However, if the number of blocks is too small, since the block size of the square matrix is decomposed too large, the amount of calculation data in the GPU increases too much, and thus the input matrix calculation processing time may increase.

In addition, when the number of blocks (# of Block) is large, it means that the block size of the square matrix is decomposed relatively small, and it may mean that the number of exchanges of block data between a plurality of GPUs increases. Accordingly, as the number of data exchanges between the plurality of GPUs increases, the processing time of the input matrix multiplication operation in the plurality of GPUs may increase.

For example, in FIG. 6, when the number of blocks of a square matrix is decomposed into about 50, the number of blocks may be optimal. In this case, the data may mean evenly distributed over a plurality of GPUs. Therefore, the speed of input matrix operation may have an optimal value.

According to an embodiment of the present disclosure, in the parallel matrix multiplication operation using a plurality of GPUs described in FIG. 3, when the square matrix C is decomposed into a plurality of blocks in the square matrix C = AxB, the parallel matrix multiplication operation ( A function for estimating an optimal block size (b*) for parallel matrix multiplication) may use b*=g(n,s). In this case, n is the size of the square matrix, and s may mean the number of GPUs performing operations on one CPU. The function g may be a function that infers an optimal block size (b*) using machine learning such as deep learning and regression analysis.

Regression analysis is an analysis method that measures the degree of fit after obtaining a model between two variables for observed continuous variables. For example, Gaussian process regression may be used. In addition, data analysis through machine learning using an artificial neural network can obtain statistically high accuracy. In the present disclosure, the optimal block size was estimated by applying Gaussian regression analysis, but this is only an example and is not limited thereto.

For example, machine learning such as deep learning may infer and extract a hidden key value from a database such as big data. That is, deep neural networks (DNNs) may calculate input values, repeat training, and infer key values obtained as a result of training.

According to an embodiment of the present disclosure, input values to the deep neural network (DNN) may be the size (n) of a square matrix in a square matrix C = AxB, the number (s) of GPUs connected to the CPU, and the block size (b). . In this case, the testing function for extracting the optimal block size (b*) may be b*=arg _b min f (s,n,b). Here, f (n, s, b) may be the execution time of the parallel matrix multiplication function f . The system 10 may extract the function b*=g(n,s) based on the values (n,s,b) input to the DNN by using a deep learning machine learning method and regression analysis. Thus, the system 10 can infer the h-dimensional vector, which is the number of hidden nodes in the DNN.

7 is a table showing results of estimating an optimal block size according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, a database may be created by logging the function f (n, s, b). As described above, n may be the size of the input matrix of the GPU, s may be the number of GPUs included in the CPU, and b may be the block size in the matrix multiplication operation. For example, n may be an input matrix having a size of 4,000 (4k), 8,000 (8k), or 16,000 (16k). s may be the number of GPUs, such as 2, 3, or 4. b may range from 1k, 2k, 4k, 8k, and 16k. For example, one CPU includes two GPUs, the size of a square matrix input through the two GPUs is 16,000 (16k), and the input square matrix is decomposed into blocks of 4,000 (4k) x 4,000 (4k). (decomposition) and distributed to each GPU.

The table of FIG. 7 shows the average and standard deviation of the parameters of the function f (n, s, b) obtained by repeating the matrix multiplication operation 10 times. The result value of each item in the table is the average and standard deviation of the function f (n, s, b) performed more than 5 times at specific s, n, and b values (numbers in parentheses are standard deviations, numbers outside parentheses are averages) )am. Numbers circled in the table mean the average values at n, s, and b where the optimal block size (b*) was obtained. Based on the data obtained from the table, the computing system 10 may generate metadata and apply Gaussian Process Regression to the metadata values.

Therefore, when a parallel matrix multiplication operation is performed on a plurality of GPUs included in the CPU using the above method, the parallel matrix is partitioned into a plurality of blocks based on block decomposition on the plurality of GPUs Then, by estimating an optimal block size, a matrix having an optimal block size may be distributed to a plurality of GPUs. In this case, neural network machine learning and regression analysis (eg, Gaussian process regression) may be used to estimate the optimal block size.

According to one embodiment of the present disclosure described above, an optimal block size to be distributed to a plurality of GPUs may be estimated using machine learning. Accordingly, high-performance computing (HPC) may be provided by performing high-speed parallel calculations through a plurality of GPUs connected to the CPU.

On the other hand, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the structure of data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes storage media such as magnetic storage media (eg, ROM, floppy disk, hard disk, etc.) and optical reading media (eg, CD-ROM, DVD, etc.).

Those skilled in the art related to this embodiment will be able to understand that it can be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods are to be considered in an illustrative rather than a limiting sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

10: computing system
110: first CPU, 210: second CPU
120: first connector (root complex) 220: second connector (root complex)
130: first memory 23: second memory
140: first GPU, 150: second GPU, 240: third GPU, 250: fourth GPU

Claims (12)

In a computing system for multi-process,
CPU (Central Processing Unit);
A plurality of GPUs (Graphics Processing Units) connected to the CPU; and
A memory included in each of the plurality of GPUs; includes,
Each of the plurality of GPUs performs a matrix multiplication operation having a specified block size in the memory of each GPU,
The designated block size is a value estimated using machine learning based on the number of a plurality of GPUs connected to the CPU and the size of a square matrix obtained through the matrix multiplication operation of the plurality of GPUs Computing system. According to claim 1,
One of the plurality of GPUs is a master and the other GPUs are slaves, and the master GPU stores the input matrix size of each of the plurality of GPUs, the number of the plurality of GPUs, and the block size as a database in the memory of the master GPU. Computing system to build. According to claim 2,
The master GPU,
A computing system for dividing the square matrix into the estimated block size and distributing it to the master GPU and the slave GPU. According to claim 1,
The testing function for extracting the optimal block size (b*) using the machine learning is b*=arg _b min f (s,n,b),
Here, b* is the optimal block size, f (n,s,b) is the execution time of the parallel matrix multiplication function f , s is the number of GPUs connected to the CPU, n is the size of the input square matrix, and b is the block Sized computing system. According to claim 4,
The function for estimating the block size for performing the shortest matrix multiplication operation through the machine learning,
b*= g(n,s), and
Here, g is a computing system that is a function that infers an optimal block size (b*) using machine learning and regression analysis. According to claim 1,
Further comprising a network interface connecting each of the plurality of GPUs,
The matrix multiplication operation,
Based on the address stored in the memory of each of the plurality of GPUs, each of the plurality of GPUs is performed by accessing the memory of different GPUs through the network interface. In the computing method for multi-process,
Estimating a block size to be distributed to a plurality of GPUs connected to the CPU; and
Performing a matrix multiplication operation having the estimated block size in the memory of each of the plurality of GPUs; Including,
The estimated block size is a value estimated using machine learning based on the number of a plurality of GPUs connected to the CPU and the size of a square matrix obtained through the matrix multiplication operation of the plurality of GPUs Computing method. According to claim 7,
One of the plurality of GPUs is a master and the other GPUs are slaves,
The estimating step is
Computing method further comprising: constructing a database of the input matrix size of each of the plurality of GPUs, the number of the plurality of GPUs, and the block size in a memory of the master GPU. According to claim 8,
The master GPU splitting the square matrix into the estimated block size and distributing it to the master GPU and the slave GPU; Computing method further comprising. According to claim 7,
The testing function for extracting the optimal block size (b*) using the machine learning is b*=arg _b min f (s,n,b),
Here, b* is the optimal block size, f (n,s,b) is the execution time of the parallel matrix multiplication function f , s is the number of GPUs connected to the CPU, n is the size of the input square matrix, and b is the block A computing method that is size. According to claim 10,
The function for estimating the block size for performing the shortest matrix multiplication operation through the machine learning,
b*= g(n,s), and
Here, g is a computing method that is a function of inferring an optimal block size (b*) using machine learning and regression analysis. According to claim 7,
The matrix multiplication operation,
Based on the address stored in the memory of each of the plurality of GPUs, each of the plurality of GPUs is performed by accessing the memory of each other GPU through a network interface.

Images