KR20210014897A - Matrix operator and matrix operation method for artificial neural network - Google Patents

Abstract

The present invention can provide a matrix operator and a matrix operation method for an artificial neural network. The matrix operator comprises: a first buffer for receiving and storing a first matrix which is a multiplicand matrix; a second buffer for receiving and storing a second matrix which is a multiplier matrix that is multiplied in the first matrix; and an operation unit, which receives a plurality of elements sequentially selected in column units from the first matrix, receives a plurality of elements sequentially selected in row units from the second matrix in correspondence to a column selected from the first matrix, multiplies each of the elements in the column selected from the first matrix by all elements in a row selected from the second matrix, and cumulatively adds multiplication operation results between the columns in the first matrix and the rows in the second matrix, having been sequentially selected, so as to acquire a result matrix which is a matrix multiplication operation result of the first matrix and the second matrix. According to the present invention, it is possible to increase operation efficiency and operation speed and reduce power consumption.

Description

Matrix operator and matrix operation method for artificial neural networks {MATRIX OPERATOR AND MATRIX OPERATION METHOD FOR ARTIFICIAL NEURAL NETWORK}

The present invention relates to an artificial neural network module and a scheduling method thereof, and to an artificial neural network module for highly efficient computation and a scheduling method thereof.

Recently, artificial neural networks, which are configured to process various information in a manner similar to that of the brain by simulating the way the human brain recognizes patterns, has been applied and used in various fields.

Such artificial neural networks require learning based on vast amounts of data, and in this process, a large amount of addition and multiplication operations must be performed. Accordingly, in the chip structure that performs operations for artificial neural networks, the multiply-accumulate operater (MAC) and The same multiple operation circuits must be provided.

Therefore, in recent years, a new kind of hardware accelerator field specialized in deep learning of artificial neural networks is attracting great attention. Deep learning accelerators were presented in different forms depending on the usage environment and purpose. For example, GPUs (Graphics Processing Unit) are mainly used for servers or workstations that value performance, and field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs) are used for edge devices such as smartphones that prioritize low power. Dedicated hardware designed by doing so, that is, NPU (Neural Processing Unit) is mainly used.

However, many accelerators available to date lack the flexibility to cope with various types of layers or tensors used in various artificial neural networks due to the characteristics of dedicated hardware. This drawback has a problem in that it is difficult to cope with the deep learning applications and models currently used in a wide variety.

On the other hand, in order to variably use a number of computing devices, the control circuit becomes complicated. Accordingly, some computing devices are not used and remain in an idle state in the process of performing the calculation of the artificial neural network, resulting in inefficiency. Additional power may be consumed.

Korean Patent Publication No. 10-2019-0055447 (published on May 23, 2019)

An object of the present invention is to provide a matrix operator and a matrix operation method capable of increasing operation efficiency and reducing power consumption by simultaneously performing a multiplication operation and an addition operation in parallel according to a pipeline technique.

A matrix operator for an artificial neural network according to an embodiment of the present invention for achieving the above object includes: a first buffer for receiving and storing a first matrix, which is a multiplicand matrix; A second buffer for receiving and storing a second matrix, which is a multiplier matrix multiplied by the first matrix; And receiving a plurality of elements sequentially selected in column units in the first matrix, and receiving a plurality of elements sequentially selected in row units in the second matrix corresponding to the columns selected in the first matrix, and in the first matrix The first matrix and the second matrix are multiplied by multiplying each element of the selected column with all the elements of the row selected in the second matrix, and the result of the multiplication operation between the columns of the first matrix and the rows of the second matrix that are sequentially selected. And an operation unit that obtains a result matrix that is a result of a matrix multiplication operation of the matrix.

The operation unit receives and multiplies each of the elements of the column selected in the first matrix and all the elements in the row selected in the second matrix, and multiplies the result of multiplication between the elements to the accumulated value of the previous multiplication result. It may include a plurality of processing lanes for obtaining elements of a row of a partial accumulation matrix by adding.

While each of the plurality of processing lanes adds the result of the inter-element multiplication to the accumulated value of the previous multiplication result, the element of the column in the first matrix and the row in the second matrix selected next according to a predetermined sequence. All elements are applied and multiplication operation can be performed.

Each of the plurality of processing lanes includes a plurality of process elements, and each of the plurality of process elements corresponds to one of a corresponding element in a selected column of the first matrix and a plurality of elements in a row selected in the second matrix. A multiplier for multiplying by receiving one element; An adder for updating the accumulated value by adding the multiplication result output from the multiplier to the accumulated value obtained by accumulating the multiplication result of the previously applied element; And an accumulation register that stores the accumulated value updated by the adder.

When the i-th column of the first matrix (where i is a natural number) is selected from the first buffer, the second buffer may select the i-th row of the second matrix.

The matrix calculator is implemented as an artificial neural network module for performing a designated operation on at least one layer among a plurality of layers of the artificial neural network, and the first matrix is a feature map applied to the at least one layer, and the second matrix May be a kernel defined in the at least one layer.

A method for calculating a matrix for an artificial neural network according to another embodiment of the present invention to achieve the above object includes: receiving and storing a first matrix as a multiplicand and a second matrix as a multiplier matrix multiplied by the first matrix; Receiving a plurality of elements sequentially selected in column units in the first matrix and a plurality of elements sequentially selected in row units in the second matrix corresponding to columns selected in the first matrix; Multiplying each element of the column selected in the first matrix with all the elements in the row selected in the second matrix; And obtaining a result matrix that is a result of a matrix multiplication operation of the first matrix and the second matrix by cumulatively adding a multiplication operation result between the columns of the first matrix and the rows of the second matrix that are sequentially selected.

Accordingly, the matrix calculator and the matrix calculation method for an artificial neural network according to an embodiment of the present invention enable a matrix multiplication operation and an addition operation performed in multiple layers of the artificial neural network to be simultaneously performed in parallel, thereby increasing computation efficiency and power It can reduce consumption. In particular, it is possible to maximize operation efficiency by accumulating addition operations while multiplication operations are being performed according to the pipeline technique.

1 shows a schematic structure of an example of an artificial neural network.
2 is a diagram showing a general matrix multiplication algorithm.
3 shows the number of times of multiplication and addition operations required in the matrix multiplication algorithm of FIG. 2.
4 shows a schematic structure of a matrix operator according to an embodiment of the present invention.
5 shows a detailed configuration of an operation processing lane of FIG. 4.
6 shows a detailed configuration of the process element of FIG. 5.
7 shows a matrix multiplication algorithm according to an embodiment of the present invention.
8 shows a method for calculating a matrix according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the implementation of the present invention, reference should be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the accompanying drawings. However, the present invention may be implemented in various different forms, and is not limited to the described embodiments. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit", "... group", "module", and "block" described in the specification mean units that process at least one function or operation, which is hardware, software, or hardware. And software.

1 shows a schematic structure of an example of an artificial neural network.

1 is a representative example of an artificial neural network, and illustrates a convolution neural network (CNN). In particular, it is an artificial neural network used for optical character reader in convolutional neural networks, and shows the general structure of LeNet-5, a representative convolutional neural network developed for postal code recognition and number recognition.

LeNet-5, for example, receives a 32 X 32 input image, repeatedly performs convolution and sub-sampling operations, extracts a feature map (f.map), and extracts it from the feature map. It is configured to select a value corresponding to the most probable class among the predefined classes based on the specified feature value.

Since LeNet-5 is a neural network developed to recognize numbers, LeNet-5 classifies feature values into numbers between 0 and 9 as an example, and can select one of the numbers between 0 and 9 as a result value.

Hereinafter, LeNet-5 will be described as an example, but other convolutional neural networks and artificial neural networks also have basically similarities, and the concept of the present invention is not limited to LeNet-5 and can be applied to various artificial neural networks.

In FIG. 1, C denotes a convolution layer, S denotes a sub-sampling layer, FC denotes a fully-connected layer, and numbers after C, S, and FC denote layers. Represents the index. That is, LeNet-5 includes three convolutional layers (C1, C2, C3), two sub-sampling layers (S1, S2), and two fully connected layers (FC1, FC2).

In the case of a CNN as shown in FIG. 1, each of a plurality of convolutional layers (C1, C2, C3) receives a feature map (f.map) (or an input image (Input)), and the applied feature map (f. .map) (or the input image (Input)) corresponding to each of the convolutional layers (C1, C2, C3) a predetermined kernel and a convolution operation is performed. And the convolution operation consists of a number of multiplication and addition operations. In addition, a multiplication operation is also performed in a plurality of fully connected layers FC1 and FC2.

That is, in the case of the CNN shown in FIG. 1, a number of multiplication and addition operations must be performed. In addition, in the case of artificial neural networks other than CNN, it is basically configured to perform multiplication and addition operations.

In general, artificial neural networks perform matrix multiplication and matrix addition operations by converting a feature map (or input image) and a kernel into a matrix according to a predetermined algorithm such as im2col (image blocks into columns). By doing so, the calculation speed is improved.

As is well known, the inter-matrix multiplication operation obtains an operation result by multiplying specified elements in each matrix and then adding all the multiplied results. In this case, in the case of inter-element multiplication, while the multiplication operation is performed once in parallel, the process of adding the inter-element multiplication result must be repeated a plurality of times according to the number of values to be added.

FIG. 2 is a diagram illustrating a general matrix multiplication operation method, and FIG. 3 shows the number of multiplication operations and addition operations required in the matrix multiplication operation method of FIG. 2.

As shown in FIG. 2, the multiplication operation of the matrix is performed by the elements of each row (#1, #2, #3, ..., #m) of matrix A of size m × k and B of size k × n. One element of matrix C, which is the result of multiplying matrix A and matrix B, by multiplying the corresponding elements among the elements of each column of the matrix (&1, &2, &3, ..., &n) and adding the multiplied values (# , &).

Here, the matrix A is called a multiplicand matrix, and the matrix B is called a multiplier matrix.

Referring to FIG. 3, an operation for obtaining the elements (#1, &1) of the 1st row and 1st column of the C matrix is read, the matrix A, which is the multiplicand, in row units, according to the general matrix operation rule, and the matrix B, which is the multiplier matrix. By reading in column units, the rows of the read matrix A and the columns of the matrix B are multiplied with each other, and all the multiplied results are added.

For example, first, the elements in the first row (#1) of matrix A and the elements in the first column (&1) of matrix B are multiplied with each other. Here, as an example, it is assumed that k is 16, and if the matrix operator has 16 processing elements (PE), 16 elements in the first row (#1) of matrix A and matrix B are A multiplication operation may be performed on 16 elements in the first column <RTI ID=0.0>(&1)</RTI> in parallel. That is, the multiplication of the elements in the first row (#1) of matrix A and the elements in the first column (&1) of matrix B is performed only once in parallel.

However, the sum of the 16 multiplication values obtained by multiplying the corresponding elements thereafter is not calculated in one operation. In general, an operation element such as a process element PE is configured to receive two inputs and perform a multiplication or addition operation. Therefore, as shown in FIG. 3, it is necessary to select 16 multiplication values by two and perform an addition operation first, and then to repeatedly perform an addition operation by selecting two each of the added values. This means that four addition operations must be repeatedly performed for 16 multiplication values, and as a result, the operation time of the addition operation for the multiplication result is very long compared to the inter-element multiplication operation.

In addition, since the result of inter-element multiplication must be added, inter-element multiplication and addition operations are performed sequentially. In other words, the multiplication operation and the addition operation must be performed separately. Therefore, as shown in FIG. 3, a total of five operations are required for multiplication between 16 elements.

And this is an operation to obtain a value for one element of the C matrix (for example, c _0,0 ). Assuming that the number of processing lanes including 16 process elements (PE) is m , As for the entire matrix C, the number of times proportional to the size of matrix A and matrix B must be performed, resulting in n × 5 operations.

Therefore, it is very important to reduce the time required for the addition operation in order to accelerate the operation speed of the matrix.

FIG. 4 shows a schematic structure of a matrix operator according to an embodiment of the present invention, FIG. 5 shows a detailed configuration of an operation processing lane of FIG. 4, and FIG. 6 shows a detailed configuration of a process element of FIG. 5.

Referring to FIGS. 4 to 6, the matrix operator 100 according to the present embodiment will be described. The matrix operator 100 includes an operation control unit 110, a first buffer unit 120, and a second buffer unit 130. And an operation unit 140, and, as described above, may be used as a module of an artificial neural network.

The operation control unit 110 receives a plurality of matrices to be performed in each layer of the artificial neural network. Here, the plurality of matrices to be performed may be at least one feature map (or input image) applied to a layer and at least one kernel designated for each layer.

The operation control unit 110 selects two matrices to be operated from among the applied matrices, and transfers the selected two matrices to the first buffer unit 120 and the second buffer unit 130 together with an operation command. do. At this time, the matrix A, which is a multiplicand matrix for at least one feature map applied to the layer of the artificial neural network, is applied to the first buffer unit 120, and the multiplier for the kernel designated in the layer of the artificial neural network is applied to the second buffer unit 130. Apply the matrix B, which is a matrix.

Further, the operation control unit 110 may receive the result of performing the matrix operation from the operation unit 140 and transmit it to a memory (not shown) and store it.

The operation control unit 110 applies the selected matrix together with the operation command in order to perform a matrix multiplication operation by selecting an element of each matrix based on the matrix multiplication algorithm according to the present embodiment to be described later.

The first buffer unit 120 selects an element in a column unit from the matrix A applied from the operation control unit 110 according to an operation command, and transmits it to the operation unit 140. In addition, the second buffer unit 130 selects an element in row units from the applied B matrix according to the operation command and transmits the selected element to the operation unit 140.

The operation unit 140 may include a plurality of operation processing lanes SIMDL. In addition, each of the plurality of processing lanes SIMDL may include a plurality of process elements PE and a SIMD unit SIMDU, as shown in FIG. 5.

In recent years, matrix calculators generally use SIMD (Single Instruction Multiple Data) techniques to batch-process complex operations into a single instruction in order to increase operation efficiency. The SIMD technique is a technique mainly used in vector processors in a manner in which a plurality of process elements (PEs) apply the same (or similar) operation to a plurality of data and process them at the same time.

In the SIMD technique, in order to maximize the efficiency of the instruction, multiple instruction sets that can process multiple data with a single instruction are stored. In addition, each of the stored instruction sets performs an operation in parallel using data level parallelism (DLP) for a plurality of process elements PE. That is, the SIMD unit (SIMDU) allows the first and second buffer units 120 and 130 to perform the same operation in parallel in which a plurality of process elements PE are assigned to elements applied in a row or column unit.

Here, the SIMD unit (SIMDU) may be implemented as hardware in the operation unit 140, but may be implemented as software that designates an operation performed by the operation unit 140. Also, in some cases, it may be implemented in the operation control unit 110.

Each of the plurality of process elements PE receives elements to be computed from among the elements (a, b) of matrix A and matrix B from the first and second buffer units 120 and 130 and performs multiplication or addition operation. do. In addition, each of the plurality of process elements PE may be implemented with a multiply-accumulate operater (MAC).

Referring to FIG. 6, each of the plurality of process elements PE may include a multiplier MUL, an adder ADD, and an accumulation register ACC.

The multiplier (MUL) multiplies the element (a) of the matrix A applied from the first buffer unit 120 and the element (b) of the matrix B applied from the second buffer unit 130 to output to the adder ADD. . The adder ADD updates the accumulated subtotal by receiving and adding the output value of the multiplier MUL and the previously calculated accumulated subtotal stored in the accumulation register ACC. The accumulation register ACC stores the updated accumulated subtotal output from the adder ADD.

7 shows a matrix multiplication algorithm according to an embodiment of the present invention.

Hereinafter, an algorithm for multiplying the matrix of FIG. 7 will be described with reference to FIGS. 4 to 6.

In a general matrix multiplication operation, as shown in FIG. 2, elements are selected in column units (&1, &2, ..., &k) from matrix A, which is a multiplicand matrix, and column units (&1, By selecting elements with &2, ..., &n), the corresponding elements among the selected elements are multiplied with each other, and then all of them are added to perform a multiplication operation.

On the contrary, in the matrix multiplication operation method shown in FIG. 7 according to the present embodiment, the first buffer 120 selects elements in column units (&1, &2, ..., &k) from the matrix A, which is a multiplicand matrix, The second buffer 130 selects elements in row units (#1, #2, ..., #k) from matrix B, which is a multiplier matrix, and transmits them to the operator 140.

Each of the plurality of operation processing lanes SIMDL of the operation unit 140 receives corresponding elements from matrix A and matrix B, multiplies each other, and accumulates and adds the multiplied results.

In particular, in this embodiment, each of the plurality of operation processing lanes (SIMDL) is one of the elements selected in column units (&1, &2, ..., &k) of matrix A and row units of matrix B (#1, # 2, ..., #k) are multiplied with each other.

For example, when the first column (&1) of matrix A and the first row (#1) of matrix B are selected, the first processing lane among the plurality of processing lanes (SIMDL) is the first column (&1) of matrix A. _Apply element a (a _0,0 ) in the first row of and b elements (b _0,0 , b _0,1 , ..., b _0,n ) in the first row (#1) of matrix B And multiply each of the elements a (a _0,0 ) by the elements b (b _0,0 , b _0,1 , ..., b _0,n ).

At this time, in each of the plurality of process elements PE of the first processing lane, the multiplier MUL is an element a (a _0,0 ) and elements b (b _0,0 , b _0,1 , ..., b _The corresponding b element among _0,n ) is applied, multiplied, and transferred to the adder ADD. Since there is no previously calculated multiplication result, that is, there is no accumulated value previously stored in the accumulation register ACC, the adder ADD transfers the output of the multiplier MUL as it is to the accumulation register ACC and stores it.

That is, the first processing lane is the elements (a _0,0 ) of the first row and the first column of matrix A and the elements of the first row (#1) of the matrix B (b _0,0 , b _0,1 , .. The multiplication result between ., b _0,n ) is the element value (c ⁰ _0,0 , c ⁰ _0,1 , ..., c ⁰ ) of the first row (#1) of the first cumulative matrix (C ⁰ ) _It is obtained as _0,n ). And the obtained element values (c ⁰ _0,0 , c ⁰ _0,1 , ..., c ⁰ _0,n ) of the first row (#1) are _{stored in} the accumulation register (ACC) of each process element (PE). Is saved.

Meanwhile, the second arithmetic processing lane is an element (a _1,0 ) of the second row of the first column (&1) of matrix A and the b elements (b _0,0 ,) of the first row (#1) of matrix B. b _0,1 , ..., b _0,n ) is applied, and a element (a _1,0 ) is _assigned to b elements (b _0,0 , b _0,1 , ..., b _0,n ) By multiplying each, it is acquired and stored as the element value (c ⁰ _1,0 , c ⁰ _1,1 , ..., c ⁰ _1,n ) of the second row (#2) of the first cumulative matrix (C ⁰ ) do.

In this way, the mth arithmetic processing lane is an element a (a _m,0 ) of the mth row of the first column (&1) of the matrix A and the b elements (b ₀ ) of the first row (#1) of the matrix B. _,0 , b _0,1 , ..., b _0,n ) by multiplying the element values of the mth row (#m) of the first cumulative matrix (C ⁰ ) (c ⁰ _m,0 , c ⁰ _{m, 1} , ..., c ⁰ _m,n ) and store.

That is, the multiplication of all the elements in the first column (&1) of the matrix A and all the elements in the first row (#1) of the matrix B is performed in one operation by a plurality of operation processing lanes (SIML) of the operation unit 140. It is performed simultaneously.

Thereafter, the first buffer 120 transfers the a elements (a _0,1 , a _1,1 , ..., a _m,1 ) of the second column (&2) of the matrix A to a plurality of processing lanes ( SIMDL) to the corresponding operation processing lane, and the second buffer 130 transmits the b elements (b _1,0 , b ₁ , ₁ , ..., b ₁ ) of the second row (#2) of the B matrix _,n ) are transferred to each of a plurality of operation processing lanes (SIMDL).

Accordingly, in each process element (PE) of the operation processing lane (SIMDL), one a element (a _0,1 , a _1,1 , ..., a _m ) of the second column (&2) to which a multiplier (MUL) is applied _,1 ) and the corresponding b element among the b elements (b _1,0 , b _1,1 , ..., b _1,n ) in the second row (#2) are multiplied, and the adder (ADD) is a multiplier ( MUL), the accumulated value (c ⁰ _0,0 , c ⁰ _0,1 , ..., c ⁰ _0,n ) previously acquired and stored in the accumulator register (ACC) is applied and added, 2 Obtained as the element values (c ¹ _0,0 , c ¹ _0,1 , ..., c ¹ _0,n ) of the first row (#1) of the cumulative matrix (C ¹ ), and the obtained element values ( c ¹ _0,0 , c ¹ _0,1 , ..., c ¹ _0,n ) are stored in the accumulating register (ACC) again.

Similarly, the second arithmetic processing lane is the elements (a _1,1 ) of the second row of the second column (&2) of matrix A and the elements (b _1,0 , b) of the second row of matrix B (#2). _1,0 , ..., b _1,n ) is applied, and element a (a _1,1 ) is _assigned to each of the elements (b _1,0 , b _1,0 , ..., b _1,n ) Multiply and add the result of the multiplication with the accumulated value (c ⁰ _1,0 , c ⁰ _1,1 , ..., c ⁰ _1,n ) stored in the accumulation register (ACC), and the second accumulation matrix (C ¹ ) is It is acquired and stored as the element values (c ¹ _1,0 , c ¹ _1,1 , ..., c ¹ _1,n ) of the second row (#2).

In addition, the m-th operation processing lane is an element a (a _m,0 ) in the m-th row of the second column (&2) of the matrix A and the b elements (b _1,0 ,) of the second row (#2) of the matrix B. The second cumulative matrix by multiplying b _1,0 , ..., b _{1 ,n} ) and adding the cumulative values (c ⁰ _m,0 , c ⁰ _m,1 , ..., c ⁰ _m,n ) and (c ¹⁾ stores the value obtained with the elements _{^{(c 1 m, 0, c}} 1 m, 1, ..., c 1 m, n) of the m-th row (#m) of the.

In this way, the operation unit 140 sequentially receives the elements up to the kth column (&k) of the matrix A and the kth row (#k) of the matrix B, multiplies the applied elements, and adds the previously calculated cumulative value to finally As a result of multiplying the matrix A and matrix B, the k-th cumulative matrix C ^k is obtained.

When the above matrix multiplication algorithm is described again in terms of each of the plurality of process elements PE of the plurality of processing lanes SIMDL, each of the plurality of process elements PE of the p-th SIDM lane corresponds to the corresponding first in matrix A. Each of the a elements in row p (a _p,0 , a _p,1 , ..., a _p,k ) is assigned to the corresponding b elements in the pth column in the B matrix (b _0,p , b _1,p , ..., b _k,p ) is sequentially multiplied, and the multiplication result is added to the accumulated value of the previous multiplication result, and the elements of the pth row of the kth accumulation matrix (C ^k ) (c ^k _p,0 , c ^k _{p ,0} , ..., c ^k _{p, n} ) value.

In addition, the calculation method of each of the process elements PE is the same as the process of calculating one element (#, &) of the C matrix in the general matrix multiplication operation shown in FIG. 2. However, in the case of the algorithm of Fig. 2, the operation unit performs multiplication by simultaneously applying the elements of matrix A and matrix B required to calculate one element (#, &) of matrix C, and then adds the result of the multiplication. Since the operation has to be repeatedly performed, as shown in FIG. 3, each time each element (#, &) of the C matrix is acquired, the addition operation has to be performed multiple times after the multiplication operation.

However, in the matrix multiplication algorithm according to the present embodiment shown in FIG. 7, since the cumulative matrices (C ⁰ , C ¹ , ..., C ^k ) that are sequentially accumulated are obtained, only k multiplication operations and k addition operations A A matrix C, which is a result of multiplication between a matrix and a matrix B, can be obtained.

In particular, the multiplier (MUL) in each of the plurality of process elements (PE) of the plurality of operation processing lanes (SIMDL) is the multiplication result output from the previous multiplier (MUL) and the accumulated value stored in the accumulation register (ACC). During addition, a multiplication operation can be performed by receiving elements a and b to be multiplied next. That is, a multiplier (MUL) and an adder (ADD) may perform simultaneous operations according to a pipeline technique.

This means that it takes not as much as 2k of operation time to obtain the C matrix, which is the result of the multiplication between the A matrix and the B matrix, but as much as k+1.

That is, the matrix multiplication operation time can be greatly reduced by making the matrix multiplication operation rarely require time for the addition operation.

8 shows a method for calculating a matrix according to an embodiment of the present invention.

Referring to FIGS. 4 to 7, when the matrix operation method of FIG. 8 is described, first, two matrices to be multiplied are obtained (S10 ). One of the two matrices is an m × k multiplicand is referred to as an A matrix, and the other is a k × n multiplier matrix and may be referred to as a B matrix. Here, the matrix A may be all or part of the feature map (f.map) (or input image) input to each layer of the artificial neural network, and the matrix B may be all or part of the kernel specified in each layer. have.

When two matrices to be calculated are obtained, the i-th column is selected from the matrix A, which is the multiplicand matrix (S20). Then, the ith row is selected from the matrix B, which is a multiplier matrix (S30). Here, the initial value of i is 1, and first column of matrix A and 1 row of matrix B are selected.

Then, each of the elements in the i-th column of the selected matrix A (a _0,i , a _1,i , ..., a _m,i ) is selected from all the elements in the i-th row of the selected matrix B (b _i,0 , b _{i, By} multiplying by ₁ , ..., b _{i, n} ), m × n multiplication results are obtained (S40).

When m × n multiplication results are obtained, the obtained multiplication result is added to the previously obtained accumulated value (S50). If there is no previously acquired cumulative value, the multiplication result is acquired as an initial cumulative value, and if there is a previously acquired cumulative value, the result of adding the acquired multiplication result to the previously acquired cumulative value is stored as an updated cumulative value ( S60).

Then, it is determined whether i is less than the number of columns of matrix A or k, which is the number of rows of matrix B (S70). If i is less than k (i <k), i is changed to i+1 (S80). Accordingly, the next column and the next row previously selected from the matrix A and the matrix B are selected (S20).

However, if i is greater than or equal to k, a cumulative matrix composed of the stored m × n cumulative values is output as a C matrix, which is a result of matrix multiplication of matrix A and matrix B (S90).

The method according to the present invention may be implemented as a computer program stored in a medium for execution on a computer. Here, the computer-readable medium may be any available medium that can be accessed by a computer, and may also include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, and ROM (Read Dedicated memory), RAM (random access memory), CD (compact disk)-ROM, DVD (digital video disk)-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those of ordinary skill in the art will appreciate that various modifications and other equivalent embodiments are possible therefrom.

Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

110: operation control unit 120: first buffer unit
130: second buffer unit 140: operation unit
SIMDL: processing lane PE: process element
SIMDU: SIMD unit MUL: Multiplier
ADD: Adder ACC: Accumulated register

Claims (10)

A first buffer that receives and stores a first matrix that is a multiplicand matrix;
A second buffer for receiving and storing a second matrix, which is a multiplier matrix multiplied by the first matrix; And
In the first matrix, a plurality of elements sequentially selected in column units are applied, and a plurality of elements sequentially selected in row units in the second matrix are applied in correspondence with the columns selected in the first matrix, and selected in the first matrix. The first matrix and the second matrix are multiplied by multiplying each element of a column with all the elements of the row selected from the second matrix, and the result of the multiplication operation between the columns of the first matrix and the rows of the second matrix that are sequentially selected. A matrix operator including an operator that obtains a result matrix that is a result of a matrix multiplication operation of. The method of claim 1, wherein the operation unit
Each of the elements of the column selected in the first matrix is multiplied by applying a corresponding one of the elements in the column selected in the first matrix and all the elements in the row selected in the second matrix, and the multiplication result is added to the cumulative value of the previous multiplication result A matrix operator including a plurality of processing lanes for obtaining elements of a row of an accumulation matrix. The method of claim 1, wherein each of the plurality of processing lanes
While adding the inter-element multiplication result to the accumulated accumulated value of the previous multiplication result, a multiplication operation is performed by applying elements of a column from the first matrix and all elements of a row from the second matrix selected next according to a predetermined sequence. Matrix operator to perform. The method of claim 2, wherein each of the plurality of processing lanes
It includes a number of process elements,
Each of the plurality of process elements
A multiplier for multiplying by receiving a corresponding element in a selected column of the first matrix and a corresponding element among a plurality of elements in a row selected in the second matrix;
An adder for updating the accumulated value by adding the multiplication result output from the multiplier to the accumulated value obtained by accumulating the multiplication result of the previously applied element; And
A matrix operator including an accumulation register that stores an accumulated value updated by the adder. The method of claim 1, wherein the second buffer
When an i-th column of the first matrix (where i is a natural number) is selected from the first buffer, a matrix operator that selects the i-th row of the second matrix. The method of claim 1, wherein the matrix operator
Implemented as an artificial neural network module for performing a specified operation on at least one layer among a plurality of layers of the artificial neural network,
The first matrix is a feature map applied to the at least one layer, and the second matrix is a kernel predetermined to the at least one layer. Receiving and storing a first matrix that is a multiplicand matrix and a second matrix that is a multiplier matrix that is multiplied by the first matrix;
Receiving a plurality of elements sequentially selected in column units in the first matrix and a plurality of elements sequentially selected in row units in the second matrix corresponding to columns selected in the first matrix;
Multiplying each element of the column selected in the first matrix with all the elements in the row selected in the second matrix; And
And obtaining a result matrix that is a result of a matrix multiplication operation of the first matrix and the second matrix by accumulating and adding the result of a multiplication operation between the columns of the first matrix and the rows of the second matrix that are sequentially selected. The method of claim 7, wherein obtaining the result matrix
A matrix operation method in which an element of a partial accumulation matrix is obtained by adding an inter-element multiplication result obtained in the step of multiplying with all the elements to a cumulative value obtained by accumulating a previously corresponding inter-element multiplication result. The method of claim 7, wherein each of the plurality of processing lanes
While adding the inter-element multiplication result to the accumulated accumulated value of the previous multiplication result, a multiplication operation is performed by applying elements of a column from the first matrix and all elements of a row from the second matrix selected next according to a predetermined sequence. How to perform matrix operations. The method of claim 7, wherein the step of receiving the selected plurality of elements
When an i-th column of the first matrix (where i is a natural number) is selected, a plurality of elements of the i-th row of the second matrix are applied.

Images