KR102158051B1 - Computer-enabled cloud-based ai computing service method - Google Patents

Abstract

The present invention relates to a computer-executable cloud-based artificial intelligence operation service method, wherein (a) determining a server layout for a matrix multiplication operation based on the shape and size of each of the first and second matrices and the number of servers. Step, (b) dividing the first and second matrices into first and second block-based partitions based on the server layout, (c) distributing to the first and second block-based partitions Calculating a unit operation time by applying a matrix characteristic, and (d) predicting a total operation time required for a multiplication operation of the first and second matrices based on the unit operation time.

Description

Cloud-based artificial intelligence computing service method that can be performed by a computer {COMPUTER-ENABLED CLOUD-BASED AI COMPUTING SERVICE METHOD}

The present invention relates to a computer-executable cloud-based artificial intelligence operation service technology, and more particularly, a computer-executable cloud-based artificial intelligence operation service capable of predicting computational performance that changes according to the number of servers performing matrix multiplication. It's about how.

Recent advances in hardware and software system technology have made it possible to process large data sets that were not possible in the past. Systems are increasingly applying cloud computing environments that provide scalability and fault tolerance by reducing overhead through operational tasks to accommodate the increasing number of big data analysis applications. Cloud computing services provide unique hardware configurations for various instances, and many big data processing software platforms can use these resources in a scale-out manner.

Matrix multiplication is an important kernel task in many machine learning algorithms, but it can be a very difficult task to accurately predict the time required for the task or the number of instances required to complete the task in a distributed cloud computing environment. The matrix multiplication operation is required to predict the overhead in order to maintain cost efficiency in the cloud environment.

Korean Patent Publication No. 10-0909510 (2009.07.20)

An embodiment of the present invention is to provide a computer-executable, cloud-based artificial intelligence computation service method capable of predicting computational performance that changes according to the number of servers performing matrix multiplication.

An embodiment of the present invention is a computer-executable cloud-based artificial intelligence operation service method capable of determining a server layout by matching at least one work server according to the shapes of the first and second matrices based on the number of work servers Want to provide.

An embodiment of the present invention is a computer-executable cloud-based artificial capable of predicting the total computation time by applying the ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit computation time. We intend to provide an intelligent operation service method.

Among the embodiments, the computer-executable cloud-based artificial intelligence operation service method includes the steps of (a) determining a server layout for matrix multiplication based on the shapes of each of the first and second matrices, (b ) Dividing the first and second matrices into first and second block-based partitions based on the server layout, (c) a distribution matrix for the first and second block-based partitions Calculating a unit operation time by applying a characteristic, and (d) predicting a total operation time required for a multiplication operation of the first and second matrices based on the unit operation time.

The step (a) determines at least one job server that satisfies a specific condition of a matrix multiplication operation between the first and second matrices among a plurality of servers, and the first and second job servers based on the number of job servers. The server layout may be determined by matching the at least one job server according to the shapes of the matrices.

In the step (b), each of the first and second matrices may be divided into partitions based on the first and second blocks based on the number of the job servers.

In the step (c), the first and second block-based partitions are associated with each other and allocated to the at least one work server based on the server layout, and the unit operation is performed using the operation time in the at least one work server. Time can be calculated.

In the step (c), the unit operation is performed through a performance prediction model based on a row size and a column size for the first block-based partition allocated to the at least one task server, and a column size for a second block-based partition. Time can be calculated.

The step (c) may include determining the characteristics of the distribution matrix based on a ratio of the number of job servers included in the server layout to the number of job servers in the performance prediction model.

In step (d), the total computation time may be predicted by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit computation time.

Among the embodiments, the cloud-based artificial intelligence operation service method includes: (a) first and second matrices for multiplication operations based on the number of servers composed of a power of squares are first and second block-based partitions. Dividing by, (b) calculating a unit operation time by applying a variance matrix characteristic to the first and second block-based partitions, and (c) the first and second operation times based on the unit operation time. And predicting the total operation time required for the multiplication operation of matrices.

The disclosed technology can have the following effects. However, since it does not mean that a specific embodiment should include all of the following effects or only the following effects, it should not be understood that the scope of the rights of the disclosed technology is limited thereby.

The computer-executable cloud-based artificial intelligence operation service method according to an embodiment of the present invention determines a server layout by matching at least one work server according to the shapes of the first and second matrices based on the number of work servers. I can.

The computer-executable cloud-based artificial intelligence computation service method according to an embodiment of the present invention applies a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to a unit computation time You can predict the computation time.

1 is a diagram illustrating a cloud-based artificial intelligence operation service system capable of performing a computer according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the artificial intelligence operation service device of FIG. 1.
3 is a flowchart illustrating a process of providing an artificial intelligence operation service performed by the artificial intelligence operation service device of FIG. 1.
4 is an exemplary diagram illustrating a process of performing block-based variance matrix multiplication in the artificial intelligence operation service apparatus of FIG. 1.
5 is an exemplary diagram illustrating a process of generating a performance prediction model in the artificial intelligence operation service device of FIG. 1.
FIG. 6 is an exemplary view illustrating a process of predicting a total operation time for matrix multiplication based on different server layouts in the artificial intelligence operation service apparatus of FIG. 1.

Since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and have various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only those effects, the scope of the present invention should not be understood as being limited thereto.

Meanwhile, the meaning of the terms described in the present application should be understood as follows.

Terms such as "first" and "second" are used to distinguish one component from other components, and the scope of rights is not limited by these terms. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that although it may be directly connected to the other component, another component may exist in the middle. On the other hand, when it is mentioned that a certain component is "directly connected" to another component, it should be understood that no other component exists in the middle. On the other hand, other expressions describing the relationship between the constituent elements, that is, "between" and "just between" or "neighboring to" and "directly neighboring to" should be interpreted as well.

Singular expressions are to be understood as including plural expressions unless the context clearly indicates otherwise, and terms such as “comprise” or “have” refer to implemented features, numbers, steps, actions, components, parts, or It is to be understood that it is intended to designate that a combination exists and does not preclude the presence or addition of one or more other features or numbers, steps, actions, components, parts, or combinations thereof.

In each step, the identification code (for example, a, b, c, etc.) is used for convenience of explanation, and the identification code does not describe the order of each step, and each step has a specific sequence clearly in context. Unless otherwise stated, it may occur differently from the stated order. That is, each of the steps may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer-readable codes on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices storing data that can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the field to which the present invention belongs, unless otherwise defined. Terms defined in commonly used dictionaries should be construed as having meanings in the context of related technologies, and cannot be construed as having an ideal or excessive formal meaning unless explicitly defined in the present application.

Performance prediction for matrix multiplication may be performed by calculating a time required for matrix multiplication in a cloud computing environment. That is, a method of predicting performance for multiplication between arbitrary matrices may correspond to generating a performance prediction model and predicting a time required for matrix multiplication by using the performance prediction model. The performance prediction model is generated by machine learning training data that uses the matrix characteristic that has the greatest influence on the operation time of matrix multiplication as input data and the estimated time required for matrix multiplication between matrices with the corresponding matrix characteristic as output data. May correspond to learning outcomes.

Matrix multiplication performance prediction can be performed by building a performance prediction model, which can be called a core configuration, and can be composed of a training data set generation step, a feature extraction step, and a modeling work step, and the operations performed in each step are as follows: Same as

1) Steps to create training data set

In the process of generating a training data set, matrix multiplication performance prediction may perform profiling on matrix multiplication of various shapes and sizes to construct a performance prediction model. More specifically, the matrix multiplication performance prediction may generate matrix multiplication tasks belonging to various types of matrix multiplication in order to generate training data to be used for training. For matrix multiplication performance prediction, to process matrices of all shapes and sizes, learning profiling can be performed by collecting a profile including an estimated operation time required for matrix multiplication operations between left and right matrices for a matrix multiplication operation. .

Matrix multiplication operations include multiplication between square matrices (square X square), long-thin rectangular matrices and short-wide rectangular matrices (long-thin X short-wide), and short and wide rectangular matrices depending on the shape and size of the left and right matrices. And can be broadly classified as a multiplication between a long and thin rectangular matrix (short-wide X long-thin). In addition, for matrix multiplication performance prediction, Apache Spark web UI REST API, which provides various execution indicators in JSON format, can be used to measure the computation time required for matrix multiplication, and is not necessarily limited to this, and various distributed artificial intelligence computation programs are used. Can be used.

Matrix multiplication performance prediction is performed using the NVBLAS library when performing matrix multiplication in instances that use GPU devices to obtain optimal performance for various cloud computing instances with different capacities. In the case of an instance to be used, OpenBLAS can be used. Also, for matrix multiplication performance prediction, you can use the netlib-java library to allow Spark to interact with the hardware-optimized linear algebra library. The matrix multiplication performance prediction is not necessarily limited to this, and various distributed artificial intelligence calculation programs can be used.

2) Feature extraction step

The overhead of matrix multiplication in a distributed computing environment may be affected by various resources. Matrix multiplication performance measurement can use the dimensions and products of the input matrix blocks to handle various overheads. For example, lr, lc, rc, lr*rc, lr*lc, lc* rc, lr*lc+lc*rc, and lr*lc*rc can be used as matrix properties for modeling matrix multiplication performance. Here, lr*rc represents the size of the output matrix, and lr*lr and lc*rc can represent the sizes of left and right matrix blocks that affect network overhead and I/O disk overhead, respectively. lr*lc*rc may represent the total number of multiplication operations performed in matrix multiplication.

3) Modeling work steps

In the modeling operation stage, matrix multiplication performance prediction can build a performance prediction model capable of predicting the performance of multiplying various matrices. The modeling work step may consist of a model building step and a hyper-parameter search step. For matrix multiplication performance prediction, a gradient boost (GB) regressor can be used for a model building step, and Bayesian Optimization can be used to find optimal parameters for a GB method.

The GB method is a flexible nonparametric statistical learning approach to classification and regression. The main idea of the GB method is to gradually combine several weak learners that are generally applicable only to simple linear relationships to model complex and nonlinear interactions between features. The GB model has a positive step-by-step pattern, at each step a new weak learner model is applied to the rest of the current model, and the GB model can focus more on correcting errors for previous iterations.

When building a performance prediction model, setting model parameters appropriately can be very important to improve prediction quality. Many heuristic methods, such as random walk, grid based search, and statistical inference, have been proposed to search for hyperparameters that perform best. For matrix multiplication performance prediction, a Bayesian model-based statistical inference method can be used. The Bayesian optimization method can search for a set of constituent values of the next step that can improve model quality or reduce uncertainty.

1 is a diagram illustrating a cloud-based artificial intelligence operation service system capable of performing a computer according to an embodiment of the present invention.

Referring to FIG. 1, a cloud-based artificial intelligence computing service system 100 may include a user terminal 110, an artificial intelligence computing service device 130, and a database 150.

The user terminal 110 may correspond to a computing device capable of requesting an artificial intelligence operation service such as a distributed matrix operation service from the artificial intelligence operation service device 130. The user terminal 110 may be implemented as a smartphone, a laptop computer, or a computer, but is not limited thereto, and may be implemented as various devices such as a tablet PC. The user terminal 110 may be connected to the artificial intelligence computing service device 130 through a network, and the plurality of user terminals 110 may be connected to the artificial intelligence computing service device 130 at the same time.

The artificial intelligence computing service device 130 receives a request for an artificial intelligence calculation service from the user terminal 110, and predicts the calculation time required for calculating a distributed matrix, which is essential when implementing artificial intelligence, to provide an optimal cloud computing service. It can be implemented as a server corresponding to a computer or program. The artificial intelligence operation service device 130 may be implemented as at least one cloud server operated based on distributed computing. The artificial intelligence operation service device 130 may be connected to the user terminal 110 through a wired network or a wireless network such as Bluetooth, WiFi, and the like, and may communicate with the user terminal 110 through a wired or wireless network.

The artificial intelligence operation service device 130 interlocks with the database 150 to provide a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), and a memory for at least one cloud server related to a distributed matrix operation. Resource information including a can be stored. Meanwhile, unlike FIG. 1, the artificial intelligence operation service apparatus 130 may be implemented including the database 150 inside.

The database 150 may store various pieces of information used by the AI operation service device 130 to predict the operation time of various types of variance matrix multiplication according to an AI operation service request received from the user terminal 110. . For example, the database 150 may store profiling information for various types of matrix multiplication, and may store layout information about a work server on a distributed cloud to perform matrix multiplication operations, This is not necessarily limited thereto, and information collected or processed in various forms may be stored in the process of providing an artificial intelligence operation service in order to provide an optimal cloud computing service environment for a user.

FIG. 2 is a block diagram showing the artificial intelligence operation service device of FIG. 1.

Referring to FIG. 2, the artificial intelligence operation service apparatus 130 includes a server layout determination unit 210, a partition division unit 230, a unit operation time calculation unit 250, a total operation time prediction unit 270, and a control unit ( 290) may be included.

The server layout determiner 210 may determine a server layout for a matrix multiplication operation based on the shapes of each of the first and second matrices. Here, the server layout may correspond to information on instances capable of performing a matrix multiplication operation between the first and second matrices among cloud computing instances. For example, the server layout is based on the number of available instances that can perform a matrix multiplication operation, the variance matrix operations that make up the matrix multiplication operation, and matching information between the available instances, and each variance to calculate the final matrix multiplication operation result. It may include integrated information of matrix operations. The server layout determiner 210 may determine a server layout for a matrix multiplication operation in consideration of the shape and size of each of the first and second matrices and the availability situation of a current cloud computing instance.

In one embodiment, the server layout determination unit 210 determines at least one job server that satisfies a specific condition of a matrix multiplication operation between the first and second matrices among a plurality of servers, and based on the number of job servers. The server layout may be determined by matching at least one job server according to the shapes of the first and second matrices. Here, the plurality of servers may correspond to each of the cloud computing instances, and the work server may correspond to an available cloud computing instance.

The server layout determination unit 210 may determine the size of the result matrix calculated through the matrix multiplication operation based on the size and shape of the first and second matrices, and determine the size of the result matrix, that is, the number of rows and the number of columns. You can determine the number of job servers equal to the product. For example, when the size of the result matrix having a shape of 2 rows * 2 columns is 4 (= 2 * 2), the server layout determination unit 210 may determine the number of job servers as 4, and 3 rows * 5 columns If the size of the result matrix with the shape is 15 (= 3 * 5), the number of job servers can be determined as 15. In another embodiment, the server layout determiner 210 may determine the number of work servers based on the shape or size of each of the first and second matrices regardless of the size of the result matrix.

The server layout determination unit 210 may determine a server layout by matching at least one job server according to shapes of the first and second matrices based on the number of job servers. More specifically, the server layout determination unit 210 performs a one-to-one task server to perform each of a plurality of variance matrix operations constituting the result matrix based on the shape of the result matrix determined according to the shape of the first and second matrices. Can be matched with As a result, the server layout determination unit 210 includes the number of job servers required for matrix multiplication operation, information on the distributed matrix operation performed by each job server, and location information of each job server on the cloud network. You can decide the server layout.

The partition divider 230 may divide the first and second matrices into first and second block-based partitions based on the server layout. Here, the block-based partition may correspond to a partial matrix constituting each of the left and right matrices used for matrix multiplication operations. For example, if the left or right matrix is a matrix having a size of 6 rows * 6 columns, and the row and column of the matrix are divided into two blocks, each may be divided into four block-based partitions. Each of the four divided block-based partitions may correspond to a matrix having a size of 3 rows * 3 columns. As a result, the partition divider 230 may divide the left or right matrix into two blocks for a row and two blocks for a column.

In an embodiment, the partition divider 230 may divide each of the first and second matrices into first and second block-based partitions based on the number of job servers. When the number of work servers included in the server layout is 12, the partition divider 230 divides the first and second matrices so that the matrix multiplication operation between the first and second matrices is distributed and performed by a total of 12 work servers. It can be divided into first and second block-based partitions. The shapes of the first and second block-based partitions divided by the partition dividing unit 230 may be determined based on the shapes of the first and second matrixes and the number of job servers.

In an embodiment, the partition dividing unit 230 may divide the first and second matrices for a multiplication operation into first and second block-based partitions based on the number of servers configured as powers of squares. . For example, when the number of servers available in the cloud computing environment is a power of 2, 3, and 4, respectively, such as 4, 9, and 16, the partitioning unit 230 It can be divided into 2, 3, and 4 block-based partitions for overheating. If the size of the first matrix is 16000 * 32000 and the size of the second matrix is 32000 * 16000, if the number of servers is 4, each server has the first block-based partition corresponding to the matrix of 8000 * 16000 and 16000 * 8000. A variance matrix operation between the second block-based partitions corresponding to the in matrix may be performed. The partition divider 230 may divide the first and second matrices into first and second block-based partitions on the assumption that the number of servers is equal to the power of the square for efficiency in matrix distribution.

The unit operation time calculation unit 250 may calculate a unit operation time by applying a variance matrix characteristic to the first and second block-based partitions. The unit operation time calculation unit 250 may calculate the unit operation time in consideration of the operation time in each work server responsible for calculating the variance matrix on the first and second block-based partitions, and the unit operation time is It can be calculated using a performance prediction model.

Here, the variance matrix characteristic may correspond to a matrix characteristic that may affect an operation time required for matrix multiplication between arbitrary matrices having various sizes and shapes. For example, the variance matrix properties are the row size (LR) and column size (LC) of the left matrix, the row size (LC) and column size (RC) of the right matrix, the total size of the left matrix (LR*LC), and the right Include the total size of the matrix (LC*RC), the sum of the sizes of the left and right matrices (LR*LC+LC*RC), the size of the resulting matrix (LR*RC), and the number of matrix operations (LR*LC*RC). I can.

In one embodiment, the unit calculation time calculation unit 250 associates the first and second block-based partitions with each other, allocates it to at least one work server based on the server layout, and uses the calculation time in at least one work server. Thus, the unit operation time can be calculated. The unit operation time calculation unit 250 may determine first and second block-based partitions associated with a distributed matrix operation in charge of a specific job server based on matching information included in the server layout, and corresponding first and second blocks A unit operation time may be calculated through a performance prediction model based on the shape and size of the base partitions.

In one embodiment, the unit operation time calculation unit 250 predicts performance based on the row size and column size of the first block-based partition allocated to at least one work server, and the column size of the second block-based partition. The unit calculation time can be calculated through the model. The unit operation time calculation unit 250 inputs input data including the row size and column size for the first block-based partition and the column size for the second block-based partition into the performance prediction model, and at least one task server The operation time of can be calculated, and the unit operation time can be calculated based on this.

In an embodiment, the unit operation time calculator 250 may determine a distribution matrix characteristic based on a ratio of the number of job servers included in the server layout to the number of job servers in the performance prediction model. For example, when the number of job servers in the performance prediction model is 4 and the number of job servers included in the server layout is 9, the unit operation time calculation unit 250 is 2/3 of the size of the first and second matrices, respectively. The size of the corrected first and second matrices may be determined by multiplying by. The unit operation time calculator 250 may determine the corrected row size and column size of the first matrix and the corrected column size of the second matrix as the variance matrix characteristics.

The total operation time prediction unit 270 may predict the total operation time required for the multiplication operation of the first and second matrices based on the unit operation time. The total calculation time prediction unit 270 may calculate the unit calculation time in the work server and calculate the total calculation time by integrating the calculated unit calculation times. When calculating the unit operation time through the performance prediction model, the total operation time prediction unit 270 may calculate the total operation time by using a linear equation based on the calculated unit operation time.

In one embodiment, the total computation time prediction unit 270 may predict the total computation time by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit computation time. . The total calculation time prediction unit 270 uses that the performance prediction model is generated on the assumption that only a specific number of job servers are available, and calculates the total by adjusting the ratio for the unit calculation time calculated through the performance prediction model. You can predict time. This will be described in more detail in FIG. 6.

The control unit 290 controls the overall operation of the artificial intelligence operation service device 130, and the server layout determination unit 210, the partition division unit 230, the unit operation time calculation unit 250, and the total operation time prediction unit ( 270) can manage the control flow or data flow.

3 is a flowchart illustrating a process of providing an artificial intelligence operation service performed by the artificial intelligence operation service device of FIG. 1.

Referring to FIG. 3, the artificial intelligence operation service apparatus 130 may determine a server layout for a matrix multiplication operation based on the shape of each of the first and second matrices through the server layout determination unit 210 (step S310). The artificial intelligence operation service apparatus 130 converts the first and second matrices into first and second block-based partitions based on the server layout determined by the server layout determination unit 210 through the partition division unit 230. Can be divided (step S330).

The artificial intelligence operation service apparatus 130 may calculate the unit operation time by applying the variance matrix characteristic to the first and second block-based partitions through the unit operation time calculation unit 250 (step S350). The artificial intelligence operation service apparatus 130 performs a total operation required for multiplication of the first and second matrices based on the unit operation time calculated by the unit operation time calculation unit 250 through the total operation time prediction unit 270. The time can be predicted (step S370).

4 is an exemplary diagram illustrating a process of performing block-based variance matrix multiplication in the artificial intelligence operation service device of FIG. 1.

Referring to FIG. 4, a matrix multiplication operation between arbitrary matrices A and B is divided into a plurality of divisional multiplication operations, performed on several task servers arranged on a cloud network, and then integrated into one, resulting in a matrix C being generated. You can check the process of becoming. More specifically, the left matrix A (410) is a matrix having a size of 3 rows * 2 columns, and the right matrix B (430) is a matrix having a size of 2 rows * 3 columns, and the left matrix A (410) and the right matrix B A process in which the result matrix C 450 having a size of 3 rows * 3 columns is calculated through matrix multiplication between (430) can be confirmed.

The artificial intelligence computing service device 130 can determine a total of 9 work servers (worker-1, worker-2, ..., worker-9) through the server layout determination unit 210 and for 9 work servers. The server layout in which the variance matrix operation is matched to the result matrix C 450 of 3 rows * 3 columns can be determined. The artificial intelligence operation service device 130 divides the left matrix A 410 into six first block-based partitions through the partition division unit 230 based on the server layout, and divides the right matrix B 430 into six It can be divided into second block-based partitions.

The nine job servers determined by the server layout determiner 210 may collect first and second block-based partitions through a cloud network for calculating each allocated distribution matrix. For example, the work server 1 (worker-1) may perform a variance matrix operation that calculates a value corresponding to an element of (0, 0) in the result matrix C 450. More specifically, job server 1(worker-1) is A(0, 0), A(0, 1), B(0, 1), B(1, 0) to produce C(0, 0) Can be collected. This process can correspond to cogroup work, and cogroup work time can be most affected by network performance.

When the cogroup operation is completed, each operation server may perform a matrix multiplication operation between the first and second block-based partitions and an addition operation between elements. For example, job server 1(worker-1) is A(0, 0) * B(0, 1), A(0, 1) * B(1, 0) to yield C(0, 0) A matrix multiplication operation of (A(0, 0) * B(0, 1)) + (A(0, 1) * B(1, 0)) can be performed. The matrix multiplication operation in each task server can be most affected by I/O disk performance and computing performance, and the addition operation between elements can be most affected by computing performance.

5 is an exemplary diagram illustrating a process of generating a performance prediction model in the artificial intelligence operation service device of FIG. 1.

Referring to FIG. 5, the process of generating a performance prediction model may be largely composed of a training data set generation step, a feature extraction step, and a modeling operation step.

The artificial intelligence operation service apparatus 130 may generate training data regarding matrix multiplication between matrices having various shapes and sizes in a model data generation step. In one embodiment, the artificial intelligence operation service apparatus 130 may generate a training data set for matrix multiplication through sampling from a population of various matrix multiplications.

The artificial intelligence operation service apparatus 130 may extract features of the training data set generated in the training data set generation step in the feature extraction step. More specifically, the artificial intelligence operation service apparatus 130 may extract a variance matrix characteristic for matrix multiplications in the training data set. The variance matrix characteristic may correspond to a matrix characteristic that affects an operation time required for matrix multiplication. For example, the variance matrix characteristics may include a row size (LC) and a column size (LC) of a left matrix, and a column size (RC) of a right matrix. The artificial intelligence operation service device 130 can use the unit operation time calculation unit 250 to calculate the unit operation time by calculating the distribution matrix characteristics for the first and second block-based partitions allocated to each work server. have.

The artificial intelligence operation service device 130 may generate a performance prediction model in a modeling step. The generation of a performance prediction model can be largely composed of a model building step and a hyper-parameter search step. The artificial intelligence computing service device 130 may use a gradient boost (GB) regressor in the model building step, and use Bayesian Optimization to find the optimal parameters for the GB method in the hyper parameter search step. I can. The detailed operation process of GB Regressor and Bayesian Optimization is omitted.

FIG. 6 is an exemplary view illustrating a process of predicting a total operation time for matrix multiplication based on different server layouts in the artificial intelligence operation service apparatus of FIG. 1.

Referring to FIG. 6, the artificial intelligence operation service apparatus 130 may calculate a result matrix C through matrix multiplication between a left matrix A and a right matrix B. It is assumed that the size of the left matrix A, the right matrix B, and the resulting matrix C is (48000, 24000), (24000,36000) and (48000,36000), respectively, and that the performance prediction model is variance matrix operation performed on 4 job servers. Assuming that it is generated as, the artificial intelligence operation service apparatus 130 may predict the total operation time for matrix multiplication based on different server layouts, as shown in FIG. 6. In FIG. 6, only a case in which the number of job servers included in the server layout corresponds to the power of the square is described as an example, but the artificial intelligence operation service device 130 is used even when the number of job servers included in the server layout is not The squared case with the smallest difference can be applied or a similar method can be used.

When the server layout determined by the server layout determination unit 210 includes 9 as the number of working servers, the left matrix through the partitioning unit 230 is divided into block-based partitions having a size of (16000, 8000). It can be partitioned and the right matrix can be partitioned into block-based partitions having a size of (8000, 12000). In addition, the result matrix calculated by each job server may correspond to a partial matrix having a size of (16000, 12000).

If the variance matrix operation is performed on 4 working servers with block-based partitions equal to the sizes of the block-based partitions determined based on the server layout with the above 9 working servers, the size of the left matrix is (32000, 16000) and the right The size of the matrix is (16000, 24000), and the size of the resulting matrix can be (32000, 24000). Therefore, in order to calculate the total operation time of matrix multiplication performed in 9 job servers through the performance prediction model generated based on 4 job servers, matrix characteristic information consisting of (32000, 16000, 24000) is used as a performance prediction model. It must be entered in and can be expressed as f(32000, 16000, 24000).

Here, the function f refers to the performance prediction model for matrix multiplication, and f(LR, LC, RC) is the output by inputting the row size and column size of the left matrix and the column size of the right matrix to the performance prediction model. May correspond to the result. That is, f(LR, LC, RC) may correspond to an operation time predicted by the performance prediction model for matrix multiplication between the left matrix of size LR * LC and the right matrix of size LC * RC. In addition, the final total computation time is the ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model with respect to the value of f(32000, 16000, 24000) calculated through the performance prediction model, That is, it can be calculated by multiplying it by 3/2.

를 통해 산출할 수 있고, 16개의 작업 서버들에서 분산 행렬 연산이 수행되는 경우의 전체 연산 시간은

를 통해 산출할 수 있다.As a result, assuming that the performance prediction model is generated on the assumption that the variance matrix operation is performed on 4 work servers, the total operation time when the variance matrix operation is performed on 9 work servers

It can be calculated by using

It can be calculated through

Although described above with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

100: Cloud-based artificial intelligence computing service system
110: user terminal 130: artificial intelligence operation service device
150: database
210: server layout determination unit 230: partition division unit
250: unit calculation time calculation unit 270: total calculation time prediction unit
290: control unit
410: left matrix A 430: right matrix B
450: result matrix C

Claims (8)

A method performed in an artificial intelligence operation service apparatus including a server layout determination unit, a partition division unit, a unit operation time calculation unit, and a total operation time prediction unit,
(a) determining, through the server layout determination unit, a server layout for a matrix multiplication operation based on the shapes of each of the first and second matrices;
(b) dividing the first and second matrices into first and second block-based partitions based on the server layout through the partition dividing unit;
(c) calculating a unit operation time by applying a variance matrix characteristic to the first and second block-based partitions through the unit operation time calculator; And
(d) Through the total operation time prediction unit, the matrix operation performance for cloud-based artificial intelligence, including the step of estimating the total operation time required for the multiplication operation of the first and second matrices based on the unit operation time. Prediction method.
The method of claim 1, wherein step (a)
Determine at least one job server that satisfies a specific condition of a matrix multiplication operation between the first and second matrices from among a plurality of servers, and fit the shapes of the first and second matrices based on the number of job servers. A method for predicting matrix computation performance for cloud-based artificial intelligence, characterized in that the server layout is determined by matching the at least one job server.
The method of claim 2, wherein step (b)
A method for predicting matrix computation performance for cloud-based artificial intelligence, characterized in that for dividing each of the first and second matrices into partitions based on the first and second blocks based on the number of the work servers.
The method of claim 2, wherein step (c)
The first and second block-based partitions are correlated with each other, allocating to the at least one work server based on the server layout, and calculating the unit operation time using the operation time in the at least one work server. Matrix computational performance prediction method for cloud-based artificial intelligence.
The method of claim 4, wherein step (c)
The unit operation time is calculated through a performance prediction model based on a row size and a column size for the first block-based partition allocated to the at least one job server, and a column size for a second block-based partition. Matrix computational performance prediction method for cloud-based artificial intelligence.
The method of claim 1, wherein step (c)
A matrix operation for cloud-based artificial intelligence, comprising the step of determining the characteristics of the variance matrix based on a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model. How to predict performance.
The method of claim 5, wherein step (d)
For cloud-based artificial intelligence, characterized in that the total computation time is predicted by applying a ratio of the number of job servers included in the server layout to the number of job servers of the performance prediction model to the unit computation time. How to predict matrix performance.
A method performed in an artificial intelligence operation service apparatus including a server layout determination unit, a partition division unit, a unit operation time calculation unit, and a total operation time prediction unit,
(a) dividing first and second matrices for a multiplication operation into first and second block-based partitions based on the number of servers configured as powers of squares through the partition dividing unit;
(b) calculating a unit operation time by applying a variance matrix characteristic to the first and second block-based partitions through the unit operation time calculator; And
(c) Through the total operation time prediction unit, the matrix operation performance for cloud-based artificial intelligence, including the step of predicting the total operation time required for the multiplication operation of the first and second matrices based on the unit operation time. Prediction method.

Images