KR102192560B1 - Matrix Sparsificating Method and System in Distributed Computing System - Google Patents

Abstract

The present invention relates to a method and system for sparifying a distributed computing matrix, by ensuring that a sparse matrix in a distributed computing system contains more zero components than a conventional sparse. There is an effect that it can produce calculation results faster than a distributed computing system.

Description

Matrix Sparsificating Method and System in Distributed Computing System

The present invention relates to a method of sparifying a matrix in a distributed computing system and a distributed computing system using a method of sparifying a matrix of distributed computing.

With the advent of the big data era, parallel computing methods and distributed computing methods are playing a major role in various application fields including machine learning and big data analysis. Conventional distributed computing methods are a method of distributing an operation to be processed and instructing it to be operated on several computers, and then merging operations processed on all computers to solve a huge computational problem.

However, this method has a problem that the speed of the entire operation is seriously slowed down when some of the computers on the distributed system are slowed down. As the size of the distributed system increases with the era of big data, the problem has become more fatal in the distributed system as the problem occurs frequently, and various methods have been proposed to solve this problem.

In particular, recent studies have suggested ways to overcome the above problems based on ideas obtained from coding theory. These methods are a method of efficiently distributing calculations to distributed systems, so that even if some computers do not obtain calculation output values, the entire calculation result value can be known only from the results obtained from other computers.

And, one of the other methods is the matrix sparse method. Matrix sparification refers to a method of transforming a matrix so that 0 is included in a matrix computed in a distributed computer so that the computer's computation speed is accelerated.

However, the conventionally disclosed distributed computing matrix sparification method has a problem in that the computing speed is slow because the number of zeros is limited.

It is an object of the present invention to solve the above and other problems.

Another object of the present invention is to disclose a matrix sparification method capable of performing a matrix operation faster in a distributed computing system.

; n by (k-m))이 추가되어 증가된 희소화 대상 행렬(

; n by k)이 생성되는 단계; 및 상기 증가된 희소화 대상 행렬에 계수 행렬(B; k by p)이 내적 연산되어 희소화 행렬(F; n by p)이 생성되는 단계를 포함하는 것을 특징으로 하는 분산 컴퓨팅 행렬 희소화 방법을 제공한다.In order to achieve the above or other objects, according to an aspect of the present invention, when p, k, n, and m are natural numbers, the distributed computing matrix sparse using k computing result values in a distributed computing system composed of p computers In the method, in the sparse target matrix (A; n by m), an additional matrix having the same row as the sparse target matrix (

; The sparse target matrix increased by adding n by (km)) (

; n by k) is generated; And generating a sparse matrix (F; n by p) by performing a dot product calculation of a coefficient matrix (B; k by p) on the increased sparse target matrix. to provide.

And, when t and p'are natural numbers, when the increased sparse target matrix and components of the coefficient matrix are determined, (first to t*p'th rows, A step in which components of m+1th to kth columns) are arbitrarily determined; Determining a component of a coefficient matrix based on a component of the increased sparse target matrix (first to t*p'th, first to kth); And determining a component of (t*p'+1th to nth row, m+1th to kth column) of the increased sparse target matrix by the determined coefficient matrix. have.

In addition, c is a natural number less than t, i is a natural number less than p', and d is a natural number less than kt, provided that when i is p', d is a natural number less than p-p'*(kt). In this case, the coefficient matrix is (i-1)*t+cth row, (i-1)*(kt) of the sparse matrix calculated by a dot product operation of the increased sparse target matrix and the coefficient matrix. It may be a matrix having a component that causes the component of )+d column to be 0.

In addition, in the step of determining a component of (t*p'+1th to nth, m+1th to kth) of the increased sparse target matrix, the increased sparse target matrix (T*p'+1th to nth rows, 1st to kth columns) of the sparse matrix determined by the dot product operation of the coefficient matrix and (t*p'+1th to nth rows) The component of the coefficient matrix may be determined so that the components of row n and columns 1 to p) include 0 components having a constant pattern.

In addition, a component of (t*p'+1th to nth row, m+1th to kth column) of the increased sparse target matrix is (t*p'+1th) of the sparse matrix Components of the coefficient matrix may be determined so that the components of the row to nth rows and the first to pth columns) include km 0 components for each row.

In addition, when j is a natural number less than nt*p' and e is a natural number less than km, the (t*p'+1 to nth row, m+1th column) of the increased sparse target matrix To kth column) may be a component such that the components of the t*p'+jth row and the j+e-1 (mod p)th column of the sparse matrix become 0.

In addition, according to another aspect of the present invention, the step of transforming the generated sparse matrix into a sparse transpose matrix; Distributing each row of the transformed sparse transposed matrix to the p computers; Performing a dot product operation of one row of the distributed sparse transposed matrix and an input matrix (x; n by 1) in each of the p computers; Receiving a result of a dot product operation from k computers among the p computers; And decoding the received k dot product calculation results to calculate a transpose matrix of the sparse target matrix and a dot product calculation value of the input matrix; a distributed computing method using a distributed computing matrix sparification method, comprising: Provides.

According to the distributed computing matrix sparification method according to an embodiment of the present invention, by ensuring that a sparse matrix in a distributed computing system contains more zero components than a conventional sparseization method, There is an effect that it can calculate the calculation result faster than the distributed computing system.

1 is a flowchart illustrating a distributed computing method using a distributed computing matrix sparification method according to an embodiment of the present invention.
2 is a diagram illustrating a distributed computing method using a distributed computing matrix sparse method according to an embodiment of the present invention.
3 is a flowchart illustrating a method of sparifying a distributed computing matrix according to an embodiment of the present invention.
4 is a diagram illustrating a method of sparifying a distributed computing matrix according to an embodiment of the present invention.
5 is a flowchart illustrating a method of sparifying a distributed computing matrix according to another embodiment of the present invention.
6 is a diagram illustrating a method of sparifying a distributed computing matrix according to another embodiment of the present invention.
7 is a diagram illustrating a distributed computing system using a distributed computing matrix sparification method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

As a basic premise of the present invention, it is assumed that the number of computers performing calculations on a distributed system is p, the calculation speed of k computers among p computers is high, and the calculation speed of p-k computers, which are other computers, is slow. In other words, according to the basic premise of the present invention, it is necessary to derive the desired final calculation result by merging only the result values calculated by k computers among the result values calculated by p computers that can be obtained in the entire distributed system. .

The distributed computing method and system using the distributed computing matrix sparification method according to an embodiment of the present invention is an input matrix (x; n by 1) in the distributed computing system when a sparse target matrix (A; n by m) is given. ) Is input, calculating the result of the dot product operation (A ^T x; m by 1) of the transpose matrix (A ^T ; m by n) and the input matrix (x; n by 1). The purpose.

1 is a flowchart illustrating a distributed computing method using a distributed computing matrix sparification method according to an embodiment of the present invention.

2 is a diagram illustrating a distributed computing method using a distributed computing matrix sparse method according to an embodiment of the present invention.

; n by (k-m))이 희소화 대상 행렬의 우측에 연장되어 붙여지는 방법으로 추가되어, 증가된 희소화 대상 행렬(

; n by k)이 생성된 후, 증가된 희소화 대상 행렬에 계수 행렬(B; k by p)이 내적 연산되어 희소화 행렬(F; n by p)이 생성 될 수 있다. 이에 대한 더욱 자세한 설명은 이하 도 3 내지 도6을 통하여 설명한다.Referring to FIGS. 1 to 2, when a distributed computing method using a distributed computing matrix sparification method according to an embodiment of the present invention is described, a sparse matrix may be generated using a sparse target matrix (S110). . In more detail, an additional matrix (

; n by (km)) is added in a way that extends and attaches to the right side of the sparse target matrix, thereby increasing the sparse target matrix (

; After n by k) is generated, a coefficient matrix B (k by p) is dot product calculated on the increased sparse target matrix to generate a sparse matrix F (n by p). A more detailed description of this will be described with reference to FIGS. 3 to 6 below.

In addition, the generated sparse matrix may be transformed into a sparse transposed matrix (S120). In more detail, a transpose matrix may be generated by reconfiguring the rows and columns of the sparse matrix so that they are interchanged. For example, the component value of (row 1 row) of the sparse matrix may be transformed to be a component value of (row 1 column h) in the sparse transpose matrix.

Each row of the transformed sparse transposed matrix may be distributed to the p computers (S130). In more detail, the p rows of the transformed sparse transpose matrix may be separated into one row and distributed to p computers. Thus, all of the p computers can perform operations on different rows. In FIG. 2, the sparse transposed matrices distributed to p computers are F ^T ₁ to F ^T _p It is shown as.

In each of the p computers, a dot product operation of one row of the distributed sparse transpose matrix and the input matrix (x; n by 1) may be performed (S140). In more detail, each computer to which the sparse transpose matrix is distributed may perform a dot product operation of the distributed sparse transpose matrix and the input input matrix. In this case, since all computers perform operations on different rows, the operation speed may vary depending on the computer's computing power and the component values of the rows of the distributed sparse transposed matrix.

Among the p computers, a result of the dot product operation may be received from k computers (S150). In more detail, as described above, the computation speed of the p computers performing the dot product operation of one row of the distributed sparse transpose matrix and the input matrix may be different. In this case, among the p computers, calculation result values may be received from k computers that have performed relatively fast calculations.

The received k dot product calculation results may be decoded to calculate a transpose matrix of the sparse target matrix and a dot product calculation value of the input matrix (S160). In more detail, the decoding process knows the component values of k sparse transpose matrices, input matrices, coefficient matrices, increased sparse target matrix and sparse target matrix, and knows the matrix operation in which sparse matrices are generated. Through an operation, a transpose matrix to be sparified and a dot product operation value of the input matrix may be calculated.

3 to 4 will be described in detail a method of sparifying a distributed computing matrix according to an embodiment of the present invention corresponding to the step of generating the sparse matrix of FIG. 1.

3 is a flowchart illustrating a method of sparifying a distributed computing matrix according to an embodiment of the present invention.

4 is a diagram illustrating a method of sparifying a distributed computing matrix according to an embodiment of the present invention.

; n by (k-m))이 추가되어 증가된 희소화 대상 행렬(

; n by k)이 생성될 수 있다(S310). 보다 자세하게, 도 4 (a) 내지 도 4(b)에 도시된 바 와 같이, 희소화 대상 행렬의 우측에 추가 행렬을 결합하는 방법으로 증가된 희소화 대상 행렬이 생성될 수 있다. 3 to 4, when p, k, n, and m are natural numbers, in a distributed computing matrix sparification method using k computing result values in a distributed computing system composed of p computers, a sparse target An additional matrix (A; n by m) with the same row as the target matrix (

; The sparse target matrix increased by adding n by (km)) (

; n by k) may be generated (S310). In more detail, as shown in FIGS. 4A to 4B, an increased sparse target matrix may be generated by combining an additional matrix on the right side of the sparse target matrix.

Components of the increased sparse matrix and the coefficient matrix to be computed in the dot product may be determined (S320). In more detail, as shown in FIG. 4C, the component of the coefficient matrix may have a preset fixed component value. As an example in which the component values of the coefficient matrix are determined, a coded matrix such as Reed-Solomon code widely known in code theory may be used.

A component of the increased sparse matrix may be determined by the determined coefficient matrix component (S330). In more detail, as shown in FIGS. 4 (c) to (d), the components of (first to nth rows, m+1th to kth columns) of the increased sparse target matrix The coefficient matrix of the coefficient matrix so that the components of (first to nth rows, first to pth columns) of the sparse matrix contain km 0 components in each row, and km 0 components are not arranged only in the same column. Ingredients can be determined. As an example, when q is a natural number less than n and e is a natural number less than km, the increased sparse target matrix (first to nth rows, m+1th to kth columns) The component is calculated as a component such that the components of the qth row and the q+e-1 (mod p) column of the sparse matrix become 0, so that all the component values of the increased sparse target matrix may be determined. This can be easily calculated through a linear equation. Here, q+e-1 (mod p) means the remaining value obtained by dividing q+e-1 by p.

A specific equation for obtaining the additional matrix components constituting the increased sparse target matrix is shown in Equation 1 below.

(Equation 1)

이고,

는

에 해당하는 (k-m)개의 열벡터들을 나열한 계수 행렬(B)의 부분 행열이며,

는 추가 행렬의 (제q행, 제1열 내지 제k-m열)이고,

는 추가 행렬의 (제q행, 제1열 내지 제m열)이며,

는 추가 행렬의 부분 행렬의 제1행 내지 제m행이고,

는 추가 행렬의 부분 행렬의 제m+1행 내지 제k행이다.here,

ego,

Is

It is a partial matrix of the coefficient matrix (B) listing (km) column vectors corresponding to,

Is (row q, column 1 to column km) of the additional matrix,

Is the (qth row, 1st to mth column) of the additional matrix,

Is the first to mth rows of the partial matrix of the additional matrix,

Is the m+1th row to the kth row of the partial matrix of the additional matrix.

In addition, a sparse matrix may be generated (S340). In more detail, a sparse matrix may be generated through a dot product operation of the determined increased sparse target matrix and coefficient matrix. The sparse matrix generated in this way may have k-m 0 components in each row, and the 0 components may be arranged to be evenly distributed in all columns.

5 to 6 will be described in detail a method for sparifying a distributed computing matrix according to another embodiment of the present invention corresponding to the step of generating the sparse matrix of FIG. 1.

5 is a flowchart illustrating a method of sparifying a distributed computing matrix according to another embodiment of the present invention.

6 is a diagram illustrating a method of sparifying a distributed computing matrix according to another embodiment of the present invention.

또는

중에서 작은 값을 넘지 않는 최대 자연수이고, p'은 p/(k-t) 또는 n/t 중에서 작은 값을 넘지 않는 최대 자연수이다. In the following description, when p, k, n and m are natural numbers, t is

or

Among them, it is the maximum natural number that does not exceed the smaller value, and p'is the maximum natural number that does not exceed the smaller value among p/(kt) or n/t.

보다 큰 경우, 도 6 (h)에 도시된 바와 같이 희소화 행렬의 0 성분 생성 방식에 따른 범위가 나뉠 수 있도록, 주어진 희소화 대상 행렬을 각 분할된 행렬의 행의 개수가 t*p'개가 넘지 않도록 n/(t*p') 이하의 자연수의 개수로 행을 분할한 후, 아래의 분산 컴퓨팅 행렬 희소화를 실시 한다. 만약, n이

이하인 경우, 희소화 대상 행렬을 분할하지 않고 아래의 분산 컴퓨팅 행렬 희소화를 실시 한다.n is

In the larger case, the number of rows of each partitioned matrix is t*p' so that the range according to the zero component generation method of the sparse matrix is divided as shown in FIG. 6(h). After dividing the rows by the number of natural numbers not exceeding n/(t*p'), the following distributed computing matrix sparse is performed. If n is

In the following cases, the following distributed computing matrix sparse is performed without dividing the sparse target matrix.

; n by (k-m))이 추가되어 증가된 희소화 대상 행렬(

; n by k)이 생성될 수 있다(S510). 보다 자세하게, 도 6 (a)에 도시된 바 와 같이, 희소화 대상 행렬의 우측에 추가 행렬을 결합하는 방법으로 증가된 희소화 대상 행렬이 생성될 수 있다. 5 to 6, in a distributed computing matrix sparification method using k computing result values in a distributed computing system composed of p computers, a sparse target matrix in a sparse target matrix (A; n by m) An additional matrix with rows equal to (

; The sparse target matrix increased by adding n by (km)) (

; n by k) may be generated (S510). In more detail, as illustrated in FIG. 6A, an increased sparse target matrix may be generated by combining an additional matrix on the right side of the sparse target matrix.

Some components of the increased sparse target matrix may be arbitrarily determined (S520). In more detail, as shown in Fig. 6(d), when t and p'are natural numbers, the components of (first to t*p' rows, first to kmth columns) of the additional matrix Components of (first to t*p'th rows, m+1th to kth columns) of the corresponding increased sparse target matrix may be set to arbitrary values.

A component of the coefficient matrix may be determined by components of the increased sparse target matrix (first row to t*p'th row, first column to kth column) (S530). In more detail, c is a natural number less than t, i is a natural number less than p', and d is a natural number less than kt, provided that when i is p', d is a natural number less than p-p'*(kt). In this case, the coefficient matrix is (i-1)*t+cth row, (i-1)*((i-1)*() of the sparse matrix calculated by a dot product operation of the increased sparse target matrix and the coefficient matrix. It may be a matrix having a component such that the component of column kt)+d becomes 0.

Referring to FIG. 6, as shown in FIG. 6(f), so that the components of (first to t-th rows, first to kt-th columns) of the sparse matrix are 0, FIG. 6 ( As shown in f), the component values of (first to kth rows, first to kt columns) of the coefficient matrix can be calculated, and as shown in FIG. 6(g), sparse 6(d), so that the component of the matrix (t+1th row to 2tth row, k-t+1th column to 2nd (kt) column) is 0, as shown in FIG. The component values of the first to kth rows and the k-t+1 to second (kt) columns) may be calculated. 6 (d) and (h), the component values of the coefficient matrix are included in each row up to the t*p'th row of the sparse matrix in the same manner as described above. All can be decided.

The remaining components of the increased sparse target matrix may be determined by the determined coefficient matrix (S540). In more detail, a component of (t*p'+1th to nth, m+1th to kth) of the increased sparse target matrix may be determined by the determined coefficient matrix. The (t*p'+1 to nth row of the sparse target matrix, which is increased by the same method as described in FIGS. 3 to 4 above, so that (km) 0 components can be included in each row of the matrix, The components of m+1 column to k-th column) may be determined. As a specific example, when j is a natural number less than nt*p', and e is a natural number less than km, the (t*p'+row 1 to nth, m+th of the increased sparse target matrix The components of column 1 to column k) may be calculated and determined as components such that the components of the t*p'+jth and j+e-1 (mod p)th columns of the sparse matrix are 0. Here, j+e-1 (mod p) means the remainder of j+e-1 divided by p. A detailed mathematical calculation method for this will be omitted because it overlaps with the description of FIGS. 3 to 4.

A coefficient matrix may be dot product computed on the increased sparse target matrix to generate a sparse matrix (S550). In more detail, a sparse matrix may be generated based on a result of a dot product operation of the determined increased sparse target matrix and the determined coefficient matrix.

As described above, the sparse matrix generated by the distributed computing matrix sparification method according to an embodiment of the present invention has an effect that more zero components are guaranteed than the sparse matrix generated by the conventional matrix sparification method. Accordingly, according to the method for sparging a distributed computing matrix according to an embodiment of the present invention, there is an effect that the overall distributed computing operation speed may be improved.

7 is a diagram illustrating a distributed computing system using a distributed computing matrix sparification method according to an embodiment of the present invention.

The distributed computing matrix sparse method according to an embodiment of the present invention may be performed in a distributed computing system 700 capable of performing the distributed computing matrix sparification method as described above. In more detail, the distributed computing system 700 using the distributed computing matrix sparse method includes an input matrix receiving unit 710, a sparse matrix generating unit 720, a computing unit 730, a decoding unit 740, and a control unit 750. ) Can be included.

The input matrix receiver 710 may receive an input matrix from the outside.

The sparse matrix generator 720 may generate a sparse matrix by generating an increased sparse target matrix and a coefficient matrix from the sparse target matrix, and may convert the generated sparse matrix into a transposed matrix.

The computing unit 730 may include a plurality of computers capable of performing matrix operations, and each computer may receive some rows of the transformed sparse transposed matrix to perform an input matrix and a dot product operation.

The decoder 740 may calculate a transpose matrix of a sparse target matrix and a dot product operation value of the input matrix from k result values calculated by the computing unit 730. When k or more result values are received from the computing unit 730, the decoder 740 may calculate a transpose matrix of a sparse target matrix and a dot product operation value of the input matrix.

The control unit 750 transmits the input matrix received by the input matrix receiving unit 710 to the sparse matrix generator 720 so that a sparse matrix is generated, and the sparse transpose generated by the sparse matrix generator 720 The p rows of the matrix can be distributed to each computer of the computing unit 730, and the decoding unit 740 is sparse by transmitting k result values calculated from the computing unit 730 to the decoding unit 740 The transpose matrix of the target matrix and the dot product operation value of the input matrix can be calculated.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

700: Distributed Computing System Using Distributed Computing Matrix Sparse Method
710: input matrix receiver
720: sparse matrix generator
730: computing department
740: decryption unit
750: control unit

Claims (8)

When p, k, n, m, t, and p'are natural numbers, in a distributed computing matrix sparse method using k computing results in a distributed computing system composed of p computers,
The sparse target matrix (A; n by m) has the same row as the sparse target matrix (

; The sparse target matrix increased by adding n by (km)) (

; n by k) is generated; And generating a sparse matrix (F; n by p) by performing a dot product operation of a coefficient matrix (B; k by p) on the increased sparse target matrix,
When the components of the increased sparse target matrix and the coefficient matrix are determined, components of (first to t*p'th rows, m+1th to kth columns) of the increased sparse target matrix This randomly determined step; Determining a component of a coefficient matrix based on a component of the increased sparse target matrix (first to t*p'th, first to kth); And determining a component of (t*p'+1th row to nth row, m+1th column to kth column) of the increased sparse target matrix by the determined coefficient matrix.
Distributed computing matrix sparse method comprising a.
delete The method of claim 1,
c is a natural number less than t, i is a natural number less than p', and d is a natural number less than kt, provided that when i is p', d is a natural number less than p-p'*(kt),
The coefficient matrix,
Components of the (i-1)*t+cth row and (i-1)*(kt)+d column of the sparse matrix calculated by the dot product operation of the increased sparse target matrix and the coefficient matrix are 0 A method for sparifying a distributed computing matrix, characterized in that it is a matrix having a component such that
The method of claim 1,
In the step of determining a component of (t*p'+1th row to nth row, m+1th column to kth column) of the increased sparse target matrix,
(T*p'+1th to nth rows, 1st to kth columns) of the increased sparse target matrix and the (tth) of the sparse matrix determined by the dot product operation of the coefficient matrix *p'+ 1st to nth row, 1st to pth column) components of the coefficient matrix are determined such that components of the coefficient matrix include 0 components having a constant pattern.
The method of claim 4,
Components of (t*p'+1th to nth rows, m+1th to kth columns) of the increased sparse target matrix are from (t*p'+1th rows of the sparse matrix) And a component of the coefficient matrix is determined so that the components of the nth row and the first to pth columns) include km 0 components for each row.
The method of claim 4,
When j is a natural number less than nt*p' and e is a natural number less than km,
Components of (t*p'+1th to nth rows, m+1th to kth columns) of the increased sparse target matrix,
A distributed computing matrix sparse method, characterized in that the sparse matrix is a component such that a component of the t*p'+j-th row and the j+e-1 (mod p) column of the sparse matrix becomes 0.
A distributed computing system using a distributed computing matrix sparification method including a computer on which the distributed computing matrix sparification method of any one of claims 1 and 3 to 6 is performed.
The method of claim 1,
Transforming the generated sparse matrix into a sparse transpose matrix;
Distributing each row of the transformed sparse transposed matrix to the p computers;
Performing a dot product operation of one row of the distributed sparse transposed matrix and an input matrix (x; n by 1) in each of the p computers;
Receiving a result of a dot product operation from k computers among the p computers; And
Decoding the received k dot product calculation results to calculate a transpose matrix of the sparse target matrix and a dot product calculation value of the input matrix;
Distributed computing matrix sparse method comprising a.

Images