KR20200010644A - Computer-enabled cloud-based ai computing service method - Google Patents

Abstract

The present invention relates to a computer-executable cloud-based artificial intelligence operation service method. The computer-executable cloud-based artificial intelligence operation service method comprises the steps of: (a) determining a server layout for a matrix multiplication operation based on the shape and size of each of first and second matrices and the number of servers; (b) dividing the first and second matrices into first and second block-based partitions based on the server layout; (c) calculating a unit operation time by applying dispersion matrix characteristics to the first and second block-based partitions; and (d) estimating a total operation time required for multiplication operations of the first and second matrices based on the unit operation time.

Description

Computer-based cloud-based AI computation service method {COMPUTER-ENABLED CLOUD-BASED AI COMPUTING SERVICE METHOD}

The present invention relates to a computer-implemented cloud-based AI operation technology, and more particularly, to a computer-implemented cloud-based AI operation service capable of predicting computational performance that varies with the number of servers performing matrix multiplication. It is about a method.

Recent advances in hardware and software system technology have enabled the processing of large data sets that were not possible in the past. Systems are increasingly deploying cloud computing environments that provide scalability and fault tolerance by reducing overhead through operational tasks to accommodate the growing number of big data analytics applications. Cloud computing services provide hardware configurations that are unique to various instances, and many big data processing software platforms can use these resources in a scale-out fashion.

Matrix multiplication is an important kernel task in many machine learning algorithms, but in a distributed cloud computing environment, it can be a very difficult task to accurately predict the time it takes or the number of instances required to complete it. Matrix multiplication is essential for predicting overhead in order to maintain cost efficiency in cloud environments.

Korea Patent Publication No. 10-0909510 (2009.07.20)

An embodiment of the present invention is to provide a computer-implemented cloud-based AI calculation service method that can predict the calculation performance changes according to the number of servers performing matrix multiplication.

According to an embodiment of the present invention, a computer-implemented cloud-based AI computing service method capable of determining a server layout by matching at least one job server according to shapes of first and second matrices based on the number of job servers. To provide.

According to an embodiment of the present invention, a computer-implemented cloud-based artificial machine capable of estimating the total calculation time by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit calculation time An intelligent operation service method is provided.

Among the embodiments, the computer-implemented cloud-based AI operation method includes (a) determining a server layout for matrix multiplication operation based on the shape of each of the first and second matrices, (b) Dividing the first and second matrices into first and second block based partitions based on the server layout; (c) a distribution matrix for the first and second block based partitions; Calculating a unit operation time by applying a characteristic, and (d) predicting a total operation time required for a multiplication operation of the first and second matrices based on the unit operation time.

Step (a) determines at least one job server satisfying a specific condition of a matrix multiplication operation between the first and second matrices among a plurality of servers, and based on the number of job servers, the first and second jobs are determined. The server layout may be determined by matching the at least one job server according to the shapes of the matrices.

Step (b) may be divided into partitions based on the first and second block based on the number of the job servers for each of the first and second matrices.

Step (c) associates the first and second block-based partitions with each other and allocates the at least one job server to the at least one job server based on the server layout and uses the operation time at the at least one job server to perform the unit operation. The time can be calculated.

Step (c) is the unit operation through the performance prediction model based on the size of the row and the size of the column for the first block-based partition assigned to the at least one job server and the size of the column for the second block-based partition The time can be calculated.

The step (c) may include determining the distribution matrix characteristic based on a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model.

In step (d), the total calculation time may be estimated by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model.

Among the embodiments, the cloud-based AI operation method includes (a) first and second block-based partitions for first and second matrices for a multiplication operation based on the number of servers of squared powers. (B) calculating a unit operation time by applying a dispersion matrix property to the first and second block-based partitions, and (c) the first and second units based on the unit operation time. Estimating the total operation time required for the multiplication operation of the matrices.

The disclosed technique can have the following effects. However, since a specific embodiment does not mean to include all of the following effects or only the following effects, it should not be understood that the scope of the disclosed technology is limited by this.

According to an embodiment of the present invention, a computer-implemented cloud-based AI computing service method determines a server layout by matching at least one job server according to shapes of first and second matrices based on the number of job servers. Can be.

In the computer-implemented cloud-based AI calculation method according to an embodiment of the present invention, the ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model is applied to the unit calculation time. The computation time can be predicted.

1 is a diagram illustrating a computer-implemented cloud-based AI computing service system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an artificial intelligence service device of FIG. 1.
FIG. 3 is a flowchart illustrating a process of providing an artificial intelligence calculation service performed by the artificial intelligence service apparatus of FIG. 1.
FIG. 4 is an exemplary diagram illustrating a process of performing block-based distributed matrix multiplication in the artificial intelligence service device of FIG. 1.
FIG. 5 is an exemplary diagram illustrating a process of generating a performance prediction model in the artificial intelligence computing service device of FIG. 1.
FIG. 6 is an exemplary diagram illustrating a process of estimating a total operation time for matrix multiplication on the basis of different server layouts in the apparatus for arithmetic operation shown in FIG. 1.

Description of the present invention is only an embodiment for structural or functional description, the scope of the present invention should not be construed as limited by the embodiments described in the text. That is, since the embodiments may be variously modified and may have various forms, the scope of the present invention should be understood to include equivalents for realizing the technical idea. In addition, the objects or effects presented in the present invention does not mean that a specific embodiment should include all or only such effects, the scope of the present invention should not be understood as being limited thereby.

On the other hand, the meaning of the terms described in the present application should be understood as follows.

Terms such as "first" and "second" are intended to distinguish one component from another component, and the scope of rights should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that there may be other components in between, although it may be directly connected to the other component. On the other hand, when a component is referred to as being "directly connected" to another component, it should be understood that there is no other component in between. On the other hand, other expressions describing the relationship between the components, such as "between" and "immediately between" or "neighboring to" and "directly neighboring", should be interpreted as well.

Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprise" or "have" refer to a feature, number, step, operation, component, part, or portion thereof that is implemented. It is to be understood that the combination is intended to be present and does not exclude in advance the possibility of the presence or addition of one or more other features or numbers, steps, actions, components, parts or combinations thereof.

In each step, an identification code (e.g., a, b, c, etc.) is used for convenience of description, and the identification code does not describe the order of the steps, and each step is clearly contextual. Unless stated otherwise, they may occur out of the order noted. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer readable code on a computer readable recording medium, and the computer readable recording medium includes all kinds of recording devices in which data can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. The terms defined in the commonly used dictionary should be interpreted to coincide with the meanings in the context of the related art, and should not be interpreted as having ideal or excessively formal meanings unless clearly defined in the present application.

Performance prediction for matrix multiplication may be performed by calculating the time required for matrix multiplication in a cloud computing environment. That is, a method of predicting performance for multiplication between arbitrary matrices may correspond to generating a performance prediction model and predicting time required for matrix multiplication using the performance prediction model. The performance prediction model is generated by machine learning training data whose matrix data that has the greatest influence on the computation time of matrix multiplication are input data, and the expected data for matrix multiplication between matrices having the matrix characteristics as output data. It may correspond to a learning result.

Matrix multiplication performance prediction can be performed by building a performance prediction model, which is a core configuration, and can be composed of training data set generation step, feature extraction step, and modeling work step. Same as

1) Generation of training dataset

In the generation of training data sets, matrix multiplication performance prediction may perform profiling on matrix multiplication of various shapes and sizes to build a performance prediction model. More specifically, matrix multiplication performance prediction may generate matrix multiplication tasks belonging to various types of matrix multiplication in order to generate training data to be used for learning. Matrix multiplication performance prediction can perform learning profiling by collecting a profile containing the estimated computation time for matrix multiplication operations between left and right matrices for matrix multiplication operations to process matrices of all shapes and sizes. .

Matrix multiplying works by multiplying square matrices according to the shape and size of the left and right matrices, multiplying between long and thin rectangular matrices and short and wide rectangular matrices (long-thin X short-wide) and short and wide rectangular matrices. Can be largely classified into short-wide X long-thin. In addition, matrix multiplication performance prediction may use the Apache Spark web UI REST API, which provides various performance indicators in JSON format for measuring the computation time required for matrix multiplication, and is not limited thereto. Can be used

Matrix multiplication performance prediction uses NVBLAS library and matrix CPU when performing matrix multiplication on instances that use GPU devices to achieve optimal performance for different cloud computing instances with different capacities. For instance, you can use OpenBLAS. Matrix multiplication performance prediction can also use the netlib-java library to enable Spark to interact with the hardware-optimized linear algebra library. Matrix multiplication performance prediction is not limited to this, and various distributed AI computational programs may be used.

2) Feature Extraction Step

The overhead of matrix multiplication in a distributed computing environment can be affected by various resources. Matrix multiplication performance measurements can use the dimensions and products of the input matrix blocks to handle various overheads, for example, lr, lc, rc, lr * rc, lr * lc, lc * rc, lr * lc + lc * rc and lr * lc * rc can be used as matrix properties for modeling matrix multiplication performance. Here, lr * rc represents the size of the output matrix, and lr * lr and lc * rc may represent the size of the left and right matrix blocks affecting network overhead and I / O disk overhead, respectively. lr * lc * rc may represent the total number of multiplication operations performed in matrix multiplication.

3) Modeling work steps

At the modeling stage, matrix multiplication performance prediction can build a performance prediction model that can predict the performance of multiplying the various matrices. The modeling work step may be composed of a model building step and a hyper-parameter search step. Matrix multiplication performance prediction can use the GB (Gradient Boost) regressor for the model building phase, and can use Bayesian Optimization to find the optimal parameters for the GB method.

The GB method is a flexible, nonparametric statistical learning approach to classification and regression. The main idea of the GB method is to combine several weak learners that are generally applicable only to progressively simple linear relationships in order to model complex and nonlinear interactions between features. The GB model has a forward stepped pattern, in which each new weak learner model is applied to the rest of the current model, and the GB model can focus more on correcting errors for previous iterations.

Properly setting model parameters when building a performance prediction model can be critical to improving prediction quality. Many heuristic methods, such as random walk, grid based search and statistical inference, have been proposed to search for the best performing hyperparameters. Matrix multiplication performance prediction can use statistical inference based on Bayesian models. The Bayesian optimization method can retrieve a set of configuration values of the next level that can improve model quality or reduce uncertainty.

1 is a diagram illustrating a computer-implemented cloud-based AI computing service system according to an embodiment of the present invention.

Referring to FIG. 1, the cloud-based AI computing service system 100 may include a user terminal 110, an AI computing service device 130, and a database 150.

The user terminal 110 may correspond to a computing device capable of requesting an artificial intelligence calculation service device 130 such as an arithmetic service of a distribution matrix. The user terminal 110 may be implemented as a smartphone, a notebook, or a computer, and is not limited thereto, and may also be implemented as various devices such as a tablet PC. The user terminal 110 may be connected to the artificial intelligence calculation service device 130 through a network, and the plurality of user terminals 110 may be simultaneously connected to the artificial intelligence calculation service device 130.

The AI computing service device 130 may receive an AI computing service request from the user terminal 110 and may provide an optimal cloud computing service by predicting an operation time required for a distributed matrix operation necessary when implementing AI. It may be implemented as a server corresponding to a computer or a program. The artificial intelligence computing service device 130 may be implemented as at least one cloud server operated on a distributed computing basis. The artificial intelligence computing service device 130 may be connected to the user terminal 110 through a wired network or a wireless network such as Bluetooth, WiFi, or the like, and may communicate with the user terminal 110 through a wired or wireless network.

The artificial intelligence computing service device 130 is linked to the database 150 and may include a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), and a memory for at least one cloud server associated with a distributed matrix operation. Resource information including a may be stored. On the other hand, the artificial intelligence computing service device 130 may be implemented by including a database 150 therein, unlike FIG.

The database 150 may store various pieces of information used by the AI operation apparatus 130 to predict operation time of various types of distributed matrix multiplication according to the AI operation request received from the user terminal 110. . For example, the database 150 may store profiling information for various types of matrix multiplication, store layout information about a job server on a distributed cloud to perform matrix multiplication, The present invention is not limited thereto and may store information collected or processed in various forms in the process of providing an AI computing service to provide an optimal cloud computing service environment for a user.

FIG. 2 is a block diagram illustrating an artificial intelligence service device of FIG. 1.

Referring to FIG. 2, the AI calculation service device 130 may include a server layout determiner 210, a partition divider 230, a unit operation time calculator 250, a total operation time predictor 270, and a controller ( 290).

The server layout determiner 210 may determine a server layout for a matrix multiplication operation based on the shape of each of the first and second matrices. Here, the server layout may correspond to information about instances among the cloud computing instances capable of performing a matrix multiplication operation between the first and second matrices. For example, the server layout may include the number of available instances that can perform matrix multiplication operations, the variance matrix operations constituting the matrix multiplication operation and matching information between the available instances, and each variance to yield the final matrix multiplication operation result. It may include integration information of matrix operations. The server layout determiner 210 may determine a server layout for the matrix multiplication operation in consideration of the shape and size of each of the first and second matrices and the current availability of the cloud computing instance.

In an embodiment, the server layout determiner 210 determines at least one job server that satisfies a specific condition of a matrix multiplication operation between first and second matrices among a plurality of servers and based on the number of job servers. The server layout may be determined by matching at least one job server according to the shapes of the first and second matrices. Here, the plurality of servers may correspond to each of the cloud computing instances, and the job server may correspond to an available cloud computing instance.

The server layout determiner 210 may determine the size of the result matrix calculated through the matrix multiplication operation based on the size and shape of the first and second matrices, and determine the size of the result matrix, that is, the number of rows and the number of columns. You can determine the number of job servers equal to the product of the multiplication. For example, the server layout determiner 210 may determine the number of job servers as 4 when the size of the result matrix having the shape of 2 rows * 2 columns is 4 (= 2 * 2), and the 3 rows * 5 columns If the size of the resulting matrix is 15 (= 3 * 5), the number of job servers can be determined to be 15. In another embodiment, the server layout determiner 210 may determine the number of job servers based on the shape or size of each of the first and second matrices regardless of the size of the result matrix.

The server layout determiner 210 may determine the server layout by matching at least one job server according to the shapes of the first and second matrices based on the number of job servers. More specifically, the server layout determiner 210 performs one-to-one job server to perform each of a plurality of distributed matrix operations constituting the result matrix based on the shape of the result matrix determined according to the shape of the first and second matrices. Can be matched with As a result, the server layout determiner 210 includes a number of job servers required for matrix multiplication operations, information on distributed matrix operations performed by each job server, and location information of each job server on a cloud network. You can determine the server layout.

The partition divider 230 may divide the first and second matrices into first and second block-based partitions based on the server layout. Here, the block-based partition may correspond to a partial matrix constituting each of the left and right matrices used for matrix multiplication. For example, if the left or right matrix is a matrix having a size of 6 rows * 6 columns, and the rows and columns of the matrix are divided into two blocks, the partitions may be divided into a total of four block-based partitions. The divided four block-based partitions may correspond to a matrix having a size of 3 rows * 3 columns. As a result, partition divider 230 may split the left or right matrix into two blocks for a row and two blocks for a column.

In one embodiment, the partition divider 230 may divide the first and second matrices into partitions based on the first and second blocks based on the number of job servers. When the number of job servers included in the server layout is 12, the partition dividing unit 230 divides the first and second matrices so that matrix multiplication operations between the first and second matrices may be performed by being distributed in a total of 12 job servers. It may be partitioned into first and second block based partitions. The shapes of the first and second block-based partitions divided by the partition dividing unit 230 may be determined based on the shapes of the first and second matrices and the number of job servers.

In one embodiment, the partition divider 230 may divide the first and second matrices for the multiplication operation into first and second block-based partitions based on the number of servers formed of squared powers. . For example, the partition divider 230 is a row of each of the first and second matrices when the number of servers available in the cloud computing environment is the squares of 2, 3, and 4, such as 4, 9, and 16, respectively. Partitions can be divided into two, three, and four block-based partitions for overheating. If the first matrix has a size of 16000 * 32000 and the second matrix has a size of 32000 * 16000, if the number of servers is four, each server has a first block-based partition and 16000 * 8000 corresponding to a matrix of 8000 * 16000. A distribution matrix operation may be performed between second block-based partitions corresponding to a matrix. The partition dividing unit 230 may divide the first and second matrices into partitions based on the first and second blocks on the assumption that the number of servers is equal to the squared value for the efficiency of matrix distribution.

The unit operation time calculator 250 may calculate the unit operation time by applying a dispersion matrix characteristic to the first and second block-based partitions. The unit operation time calculating unit 250 may calculate the unit operation time in consideration of the operation time of each work server in charge of the distribution matrix operation for the partitions based on the first and second blocks. It can be calculated using a performance prediction model.

Here, the dispersion matrix characteristic may correspond to a matrix characteristic that may affect the computation time required for matrix multiplication between arbitrary matrices having various sizes and shapes. For example, the variance matrix properties include the row size (LR) and column size (LC) of the left matrix, the row size (LC) and column size (RC) of the right matrix, the total size of the left matrix (LR * LC), and the right Include the total size of the matrix (LC * RC), the sum of the sizes of the left and right matrices (LR * LC + LC * RC), the size of the resulting matrix (LR * RC), and the number of matrix operations (LR * LC * RC). Can be.

In one embodiment, the unit operation time calculator 250 associates the first and second block-based partitions with each other and allocates the at least one work server based on the server layout and uses the compute time at the at least one work server. Unit calculation time can be calculated. The unit operation time calculator 250 may determine the first and second block based partitions associated with the distribution matrix operation that is performed by the specific job server based on the matching information included in the server layout, and the corresponding first and second blocks. Based on the shape and size of the base partitions, the unit operation time may be calculated through the performance prediction model.

In an embodiment, the unit operation time calculator 250 estimates the performance based on the size of the row and the size of the column for the first block-based partition and the size of the column for the second block-based partition. The unit calculation time can be calculated through the model. The unit operation time calculating unit 250 inputs input data including the size of the row and the size of the column for the first block-based partition and the size of the column for the second block-based partition, to the performance prediction model to input the input data into the performance prediction model. It is possible to calculate the operation time of and unit calculation time can be calculated based on this.

In an embodiment, the unit operation time calculator 250 may determine a distribution matrix characteristic based on a ratio of the number of work servers included in the server layout to the number of work servers of the performance prediction model. For example, when the number of job servers of the performance prediction model is 4 and the number of job servers included in the server layout is 9, the unit operation time calculator 250 is 2/3 of the size of the first and second matrices, respectively. The size of the corrected first and second matrices can be determined by multiplying by. The unit operation time calculator 250 may determine the size of the corrected row and column of the first matrix and the size of the corrected column of the second matrix as dispersion matrix characteristics.

The total computation time prediction unit 270 may predict the total computation time required for the multiplication operation of the first and second matrices based on the unit computation time. The total operation time predicting unit 270 may calculate the unit operation time in the job server and calculate the total operation time by integrating the calculated unit operation times. When calculating the unit operation time through the performance prediction model, the total calculation time prediction unit 270 may calculate the total calculation time by using a linear equation based on the calculated unit calculation time.

In one embodiment, the total computation time estimator 270 may estimate the total computation time by applying a ratio of the number of task servers included in the server layout to the number of task servers of the performance prediction model to the unit computation time. . The total calculation time prediction unit 270 calculates the total calculation by adjusting the ratio of the unit calculation time calculated through the performance prediction model using the fact that the performance prediction model is generated assuming that only a certain number of job servers are available. Predict the time. This will be described in more detail with reference to FIG. 6.

The control unit 290 controls the overall operation of the artificial intelligence calculation service device 130, the server layout determination unit 210, partition partitioning unit 230, unit operation time calculation unit 250, total operation time prediction unit ( 270 may manage the control flow or data flow between.

FIG. 3 is a flowchart illustrating a process of providing an artificial intelligence calculation service performed by the artificial intelligence service apparatus of FIG. 1.

Referring to FIG. 3, the AI operation apparatus 130 may determine a server layout for matrix multiplication operation based on the shape of each of the first and second matrices through the server layout determiner 210 (step). S310). The artificial intelligence service device 130 converts the first and second matrices into first and second block based partitions based on the server layout determined by the server layout determiner 210 through the partition divider 230. It may divide (step S330).

The artificial intelligence operation service device 130 may calculate the unit operation time by applying a distribution matrix characteristic to the first and second block-based partitions through the unit operation time calculator 250 (step S350). The AI operation service device 130 may perform all operations required for the multiplication operation of the first and second matrices based on the unit operation time calculated by the unit operation time calculator 250 through the total operation time predictor 270. The time can be predicted (step S370).

FIG. 4 is an exemplary diagram illustrating a process of performing block-based distributed matrix multiplication in the artificial intelligence service device of FIG. 1.

Referring to FIG. 4, a matrix multiplication operation between arbitrary matrices A and B is divided into a plurality of division multiplication operations, performed at several job servers arranged on a cloud network, and then merged into one to generate a matrix C as a final result. You can check the process. More specifically, the left matrix A 410 is a matrix having a size of 3 rows * 2 columns, the right matrix B 430 is a matrix having a size of 2 rows * 3 columns, and the left matrix A 410 and the right matrix B The process of calculating the result matrix C 450 having the size of 3 rows * 3 columns through matrix multiplication between 430 can be confirmed.

The artificial intelligence computing service device 130 may determine a total of nine work servers (worker-1, worker-2, ..., worker-9) through the server layout determiner 210 and for nine work servers. According to the result matrix C 450 of 3 rows * 3 columns, the server layout that matches the distribution matrix operation can be determined. The artificial intelligence computing service device 130 divides the left matrix A 410 into six first block-based partitions and partitions the right matrix B 430 into six through the partition divider 230 based on the server layout. It may be divided into second block-based partitions.

The nine job servers determined by the server layout determiner 210 may collect the first and second block-based partitions through the cloud network, respectively, for the assigned distribution matrix operation. For example, the work server 1 (worker-1) may perform a variance matrix operation that calculates a value corresponding to an element of (0, 0) of the result matrix C 450. More specifically, worker server 1 (worker-1) is A (0, 0), A (0, 1), B (0, 1), B (1, 0) to yield C (0, 0). Can be collected. This process may correspond to cogroup work, and cogroup work time may be most affected by network performance.

When the cogroup task is completed, each task server may perform a matrix multiplication operation between the first and second block-based partitions and an addition operation between elements. For example, work server 1 (worker-1) is A (0, 0) * B (0, 1), A (0, 1) * B (1, 0) to yield C (0, 0) A matrix multiplication operation of may be performed and an addition operation of (A (0, 0) * B (0, 1)) + (A (0, 1) * B (1, 0)) may be performed. Matrix multiplication operations at each work server can be most affected by I / O disk performance and computing performance, and addition operations between elements can be most affected by computing performance.

FIG. 5 is an exemplary diagram illustrating a process of generating a performance prediction model in the artificial intelligence computing service device of FIG. 1.

Referring to FIG. 5, the performance prediction model generation process may be largely composed of a training data set generation step, a feature extraction step, and a modeling work step.

The artificial intelligence operation service device 130 may generate training data regarding matrix multiplication between matrices having various shapes and sizes in a model data generation step. In one embodiment, the artificial intelligence arithmetic service device 130 may generate a training data set for matrix multiplication through sampling in a population for various matrix multiplications.

The artificial intelligence operation service device 130 may extract a feature for the training data set generated in the training data set generation step in the feature extraction step. More specifically, the artificial intelligence computing service device 130 may extract the variance matrix property of the matrix multiplications in the training data set. The variance matrix property may correspond to a matrix property that affects the computation time required for matrix multiplication. For example, the variance matrix characteristic may include the size LC of the left matrix, the size LC of the column, and the size RC of the column of the right matrix. The artificial intelligence computing service device 130 may calculate the dispersion matrix characteristics of the first and second block-based partitions allocated to each work server through the unit operation time calculator 250 and calculate the unit operation time. have.

The artificial intelligence computing service device 130 may generate a performance prediction model in a modeling step. Performance prediction model generation can be largely composed of a model building step and a hyper-parameter search step. The AI computing service device 130 may use a gradient boost (GB) regressor in the model building phase and may use Bayesian Optimization to find the optimal parameters for the GB method in the hyperparameter retrieval phase. Can be. The detailed operation of GB Regressor and Bayesian Optimization is omitted.

FIG. 6 is an exemplary diagram illustrating a process of estimating a total operation time for matrix multiplication on the basis of different server layouts in the artificial intelligence service apparatus of FIG. 1.

Referring to FIG. 6, the artificial intelligence computing service device 130 may calculate a result matrix C through matrix multiplication between a left matrix A and a right matrix B. The dimensions of left matrix A, right matrix B and result matrix C are (48000, 24000), (24000,36000) and (48000,36000), respectively, and the performance prediction model assumes that distributed matrix operations are performed on four job servers. If it is assumed that is generated by the AI service device 130, as shown in Figure 6, based on the different server layout, it is possible to predict the total operation time for the matrix multiplication. In FIG. 6, only the case where the number of job servers included in the server layout corresponds to the squared power is described as an example. However, even when the number of job servers included in the server layout is not the squared power, the AI calculation apparatus 130 may be used. The least squares case can be applied or a similar method can be used.

If the server layout determined by the server layout determiner 210 includes 9 as the number of working servers, the left matrix through the partition divider 230 is a block-based partition having a size of (16000, 8000). The right matrix may be divided into block based partitions having sizes of (8000, 12000). In addition, the result matrix calculated at each job server may correspond to a partial matrix having a size of (16000, 12000).

If the distribution matrix operation is performed on four work servers with block-based partitions that are identical to the sizes of the block-based partitions determined based on the server layout having the nine work servers, the size of the left matrix is (32000, 16000) and the right side. The size of the matrix can be (16000, 24000) and the size of the resulting matrix can be (32000, 24000). Therefore, in order to calculate the total computation time of matrix multiplication performed in nine work servers through a performance prediction model generated based on four work servers, matrix characteristic information consisting of (32000, 16000, and 24000) is calculated using a performance prediction model. It should be entered in f, and can be expressed as f (32000, 16000, 24000).

Here, the function f denotes a performance prediction model for matrix multiplication, and f (LR, LC, RC) denotes the size of the row and column of the left matrix and the size of the column of the right matrix into the performance prediction model. This may be the result. That is, f (LR, LC, RC) may correspond to the computation time predicted by the performance prediction model for matrix multiplication between the left matrix of LR * LC size and the right matrix of LC * RC size. In addition, the final total calculation time is a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model with respect to the value of f (32000, 16000, 24000) calculated through the performance prediction model, That is, it can be calculated by multiplying by 3/2.

를 통해 산출할 수 있고, 16개의 작업 서버들에서 분산 행렬 연산이 수행되는 경우의 전체 연산 시간은

를 통해 산출할 수 있다.As a result, assuming that the performance prediction model is generated assuming that the distributed matrix operation is performed on four work servers, the total computation time when the distributed matrix operation is performed on nine work servers is

It can be calculated through, and the total operation time when the distributed matrix operation is performed in 16 job servers

It can be calculated through

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

100: cloud-based AI computing service system
110: user terminal 130: AI computing service device
150: database
210: server layout determination unit 230: partition partition
250: unit operation time calculation unit 270: total operation time prediction unit
290: control unit
410: left matrix A 430: right matrix B
450: result matrix C

Claims (8)

(a) determining a server layout for a matrix multiplication operation based on the shape of each of the first and second matrices;
(b) dividing the first and second matrices into first and second block based partitions based on the server layout;
(c) calculating a unit operation time by applying a dispersion matrix characteristic to the first and second block-based partitions; And
and (d) estimating a total operation time required for the multiplication operation of the first and second matrices based on the unit operation time.
The method of claim 1, wherein step (a)
Determine at least one job server among a plurality of servers that satisfies a specific condition of a matrix multiplication operation between the first and second matrices and fit the shapes of the first and second matrices based on the number of job servers. And the server layout is determined by matching the at least one job server.
The method of claim 2, wherein step (b)
And computing the first and second block-based partitions into partitions based on the number of the job servers for each of the first and second matrices.
The method of claim 2, wherein step (c)
Associating the first and second block-based partitions with each other and assigning the at least one job server to the at least one job server based on the server layout, and calculating the unit operation time using the operation time at the at least one job server. Cloud-based AI computing service method that can be performed by a computer.
The method of claim 4, wherein step (c)
The unit operation time is calculated through a performance prediction model based on a row size and a column size for the first block-based partition and a column size for a second block-based partition allocated to the at least one work server. Cloud-based AI computing service method that can be performed by a computer.
The method of claim 1, wherein step (c)
And determining the distribution matrix characteristic based on a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model. Service method.
The method of claim 5, wherein step (d)
The computer-implemented cloud-based artificial tree is estimated by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit operation time. Intelligent operation service method.
(a) dividing the first and second matrices for the multiplication operation into first and second block-based partitions based on the number of servers formed of squared powers;
calculating a unit operation time by applying a dispersion matrix property to the first and second block-based partitions; And
and (c) predicting a total operation time required for a multiplication operation of the first and second matrices based on the unit operation time.

Images