TWI814618B - Matrix computing device and operation method thereof - Google Patents
Matrix computing device and operation method thereof Download PDFInfo
- Publication number
- TWI814618B TWI814618B TW111139781A TW111139781A TWI814618B TW I814618 B TWI814618 B TW I814618B TW 111139781 A TW111139781 A TW 111139781A TW 111139781 A TW111139781 A TW 111139781A TW I814618 B TWI814618 B TW I814618B
- Authority
- TW
- Taiwan
- Prior art keywords
- matrix
- weight
- weights
- column
- input data
- Prior art date
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 311
- 238000000034 method Methods 0.000 title claims abstract description 15
- 230000009466 transformation Effects 0.000 claims abstract 2
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000011017 operating method Methods 0.000 claims description 9
- 238000009825 accumulation Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 14
- 230000017105 transposition Effects 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
- G06F7/491—Computations with decimal numbers radix 12 or 20.
- G06F7/498—Computations with decimal numbers radix 12 or 20. using counter-type accumulators
- G06F7/4983—Multiplying; Dividing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
- G06F7/50—Adding; Subtracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
- G06F7/52—Multiplying; Dividing
- G06F7/523—Multiplying only
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
- G06F7/544—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices for evaluating functions by calculation
- G06F7/5443—Sum of products
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/76—Arrangements for rearranging, permuting or selecting data according to predetermined rules, independently of the content of the data
- G06F7/78—Arrangements for rearranging, permuting or selecting data according to predetermined rules, independently of the content of the data for changing the order of data flow, e.g. matrix transposition or LIFO buffers; Overflow or underflow handling therefor
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Complex Calculations (AREA)
- Structure Of Telephone Exchanges (AREA)
- Vehicle Body Suspensions (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
Description
本發明是有關於一種運算裝置用於運算裝置的操作方法,且特別是有關於一種矩陣運算裝置用於矩陣運算裝置的操作方法。 The present invention relates to a computing device and an operating method for the computing device, and in particular, to a matrix computing device and an operating method for the matrix computing device.
圖1是矩陣乘法運算的示意圖。圖1示出矩陣MA、MB。矩陣MA是具有M個列(row)以及K個行(column)的矩陣。矩陣MB是具有K個列以及N個行的矩陣。因此,矩陣MA乘以矩陣MB會產生具有M個列以及N個行的矩陣MP。 Figure 1 is a schematic diagram of matrix multiplication operation. Figure 1 shows matrices MA, MB. Matrix MA is a matrix having M columns (rows) and K rows (columns). Matrix MB is a matrix with K columns and N rows. Therefore, multiplying matrix MA by matrix MB produces matrix MP with M columns and N rows.
應注意的是,基於矩陣乘法,矩陣MA、MB的向量方向彼此不同。也就是說,矩陣MB中的元素值的讀取順序與矩陣MA中的元素值的讀取順序並不相同。一般來說,矩陣的元素值的排列順序是優先完成元素列的排列。一旦矩陣運算裝置完成單一元素列的排列,矩陣運算裝置會進行下一元素列的排列。矩陣的元素值的讀取順序是優先讀取元素列。然而,基於矩陣乘法,矩陣MB的元素值的讀取順序是優先讀取元素行。一旦矩陣運算裝置完 成單一元素行的排列,矩陣運算裝置會進行下一元素行的排列。 It should be noted that based on matrix multiplication, the vector directions of matrices MA, MB are different from each other. That is to say, the reading order of the element values in the matrix MB is not the same as the reading order of the element values in the matrix MA. Generally speaking, the arrangement order of the element values of the matrix is to complete the arrangement of the element columns first. Once the matrix operation device completes the arrangement of a single element column, the matrix operation device will arrange the next element column. The order of reading the element values of the matrix is to read the element column first. However, based on matrix multiplication, the reading order of element values of matrix MB is to read element rows first. Once the matrix operation device is completed Arrangement into a single element row, the matrix operation device will arrange the next element row.
矩陣運算裝置利用額外的轉置(transpose)工具(如電路或演算法)來對矩陣MB進行轉置運算。因此,矩陣運算裝置的成本會增加。 The matrix operation device uses additional transpose tools (such as circuits or algorithms) to perform transpose operations on the matrix MB. Therefore, the cost of the matrix operation device increases.
本發明提供一種能夠免於轉置運算的矩陣運算裝置以及操作方法。 The present invention provides a matrix operation device and an operation method that can avoid transposition operations.
本發明的矩陣運算裝置包括儲存單元、控制電路以及運算電路。儲存單元包括權重矩陣。控制電路耦接於儲存單元。控制電路依據輸出矩陣的矩陣形狀來對權重矩陣中的多個權重的排列順序進行重新定序以確定出所述多個權重的權重讀出順序。權重讀出順序不同於權重矩陣中的所述多個權重的排列順序。運算電路耦接於控制電路。運算電路基於權重讀出順序來接收所述多個權重,並對所述多個權重以及輸入資料矩陣進行矩陣運算以產生運算矩陣。控制電路對運算矩陣進行維度轉換以產生輸出矩陣,並且將輸出矩陣寫入至儲存單元。 The matrix operation device of the present invention includes a storage unit, a control circuit and an operation circuit. The storage unit includes a weight matrix. The control circuit is coupled to the storage unit. The control circuit reorders the arrangement order of the plurality of weights in the weight matrix according to the matrix shape of the output matrix to determine the weight reading order of the plurality of weights. The weight reading order is different from the arrangement order of the multiple weights in the weight matrix. The computing circuit is coupled to the control circuit. The operation circuit receives the plurality of weights based on the weight reading sequence, and performs matrix operation on the plurality of weights and the input data matrix to generate an operation matrix. The control circuit performs dimension conversion on the operation matrix to generate an output matrix, and writes the output matrix into the storage unit.
本發明的操作方法用於矩陣運算裝置。矩陣運算裝置包括儲存單元以及運算電路。操作方法包括:依據輸出矩陣的矩陣形狀來對儲存單元的權重矩陣中的多個權重的排列順序進行重新定序以確定出所述多個權重的權重讀出順序,其中權重讀出順序不同於權重矩陣中的所述多個權重的排列順序;由運算電路基於 權重讀出順序來接收所述多個權重,並對所述多個權重以及輸入資料矩陣進行矩陣運算以產生運算矩陣;以及對運算矩陣進行維度轉換以產生輸出矩陣,並且將輸出矩陣寫入至儲存單元。 The operating method of the present invention is used in a matrix operation device. The matrix operation device includes a storage unit and an operation circuit. The operation method includes: re-sequencing the arrangement order of the plurality of weights in the weight matrix of the storage unit according to the matrix shape of the output matrix to determine the weight readout order of the plurality of weights, wherein the weight readout order is different from The order of arrangement of the multiple weights in the weight matrix; determined by the operation circuit based on The weight reading sequence receives the multiple weights, performs matrix operations on the multiple weights and the input data matrix to generate an operation matrix; and performs dimension conversion on the operation matrix to generate an output matrix, and writes the output matrix to storage unit.
基於上述,矩陣運算裝置以及操作方法依據輸出矩陣的矩陣形狀來對權重矩陣中的多個權重的排列順序進行重新定序以確定出所述多個權重的權重讀出順序。運算電路基於權重讀出順序來對所述多個權重以及輸入資料矩陣進行矩陣運算以產生運算矩陣。應注意的是,權重讀出順序改變了運算矩陣的元素排列順序。運算矩陣的元素排列順序有助於在進行維度轉換時就實現了轉置效果。因此,矩陣運算裝置並不需要利用額外的轉置工具來對矩陣進行轉置運算。也因此,本發明的矩陣運算裝置的運行成本並不會被增加。 Based on the above, the matrix operation device and the operation method reorder the arrangement order of the plurality of weights in the weight matrix according to the matrix shape of the output matrix to determine the weight reading order of the plurality of weights. The operation circuit performs matrix operation on the plurality of weights and the input data matrix based on the weight reading order to generate an operation matrix. It should be noted that the weight reading order changes the order of elements of the operation matrix. The order in which the elements of the operation matrix are arranged helps achieve the transposition effect when performing dimension conversion. Therefore, the matrix operation device does not need to use additional transposition tools to perform transpose operations on the matrix. Therefore, the operating cost of the matrix operation device of the present invention will not be increased.
為讓本發明的上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明如下。 In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, embodiments are given below and described in detail with reference to the accompanying drawings.
100:矩陣運算裝置 100:Matrix operation device
110:儲存單元 110:Storage unit
120:控制電路 120:Control circuit
130、230:運算電路 130, 230: Arithmetic circuit
231(1)~231(N):乘積累加電路 231(1)~231(N): Multiply and accumulate circuit
AD:加法器 AD: Adder
CH(1)~CH(N):通道 CH(1)~CH(N): channel
E1~EN、E11~ETS:運算元素值 E 1 ~E N , E 11 ~E TS : operation element value
IN1~INM:輸入元素值 IN 1 ~IN M : Input element value
MA、MB、MP:矩陣 MA, MB, MP: matrix
MC:運算矩陣 MC: operation matrix
MI:輸入資料矩陣 MI: input data matrix
MO:輸出矩陣 MO: output matrix
MT:轉置矩陣 MT: transposed matrix
MU:乘法器 MU: Multiplier
MV:乘積值 MV: product value
MW:權重矩陣 MW: weight matrix
ORD:權重讀出順序 ORD: weight reading order
RO1:第1讀出列 RO1: The first read column
RO2:第2讀出列 RO2: The second read column
RO3:第3讀出列 RO3: The third read column
RON:第N讀出列 RON:Nth read column
RO(S+1):第(S+1)讀出列 RO(S+1): The (S+1)th read column
RG:暫存器 RG: register
S100:操作方法 S100: How to operate
S110~S130:步驟 S110~S130: steps
W11~WNM:權重 W 11 ~W NM : weight
圖1是矩陣乘法運算的示意圖。 Figure 1 is a schematic diagram of matrix multiplication operation.
圖2是依據本發明一實施例所繪示的矩陣運算裝置的示意圖。 FIG. 2 is a schematic diagram of a matrix operation device according to an embodiment of the present invention.
圖3是依據本發明一實施例所繪示的矩陣運算的示意圖。 FIG. 3 is a schematic diagram of matrix operations according to an embodiment of the present invention.
圖4A是現行的矩陣運算的簡易範例示意圖。 Figure 4A is a schematic diagram of a simple example of current matrix operations.
圖4B是依據本發明一實施例所繪示的矩陣運算的簡易範例示意圖。 FIG. 4B is a schematic diagram of a simple example of matrix operation according to an embodiment of the present invention.
圖5是依據本發明一實施例所繪示的運算電路的電路示意圖。 FIG. 5 is a schematic circuit diagram of a computing circuit according to an embodiment of the present invention.
圖6是依據本發明一實施例所繪示的操作方法的示意圖。 FIG. 6 is a schematic diagram of an operating method according to an embodiment of the present invention.
本發明的部份實施例接下來將會配合附圖來詳細描述,以下的描述所引用的元件符號,當不同附圖出現相同的元件符號將視為相同或相似的元件。這些實施例只是本發明的一部份,並未揭示所有本發明的可實施方式。更確切的說,這些實施例只是本發明的專利申請範圍中的範例。 Some embodiments of the present invention will be described in detail with reference to the accompanying drawings. The component symbols cited in the following description will be regarded as the same or similar components when the same component symbols appear in different drawings. These embodiments are only part of the present invention and do not disclose all possible implementations of the present invention. Rather, these embodiments are only examples within the scope of the patent application of the invention.
請參考圖2,圖2是依據本發明一實施例所繪示的矩陣運算裝置的示意圖。在本實施例中,矩陣運算裝置100包括儲存單元110、控制電路120以及運算電路130。儲存單元110包括權重矩陣MW。在本實施例中,權重矩陣MW例如是具有N個列以及M個行的二維矩陣(本發明並不以此為限)。權重矩陣MW包括權重W11~WNM。在本實施例中,儲存單元110可以是由本領域技術人員所熟知的記憶體元件來實現。
Please refer to FIG. 2 , which is a schematic diagram of a matrix operation device according to an embodiment of the present invention. In this embodiment, the
在本實施例中,控制電路120耦接於儲存單元110。控制電路120依據輸出矩陣MO的矩陣形狀來對權重W11~WNM的排列順序進行重新定序(re-order)以確定出權重W11~WNM的權重讀出
順序ORD。輸出矩陣MO例如是具有T個列以及S個行的二維矩陣(本發明並不以此為限)。在本實施例中,S、T分別是大於1的正整數。
In this embodiment, the
在本實施例中,在權重W11~WNM被寫入儲存單元110的過程中,權重W11~WNM是優先以列方式被寫入。也就是說,權重W11~W1M被依序寫入至權重矩陣MW的第一列。接下來,權重W21~W2M被依序寫入至權重矩陣MW的第二列,依此類推。因此,在權重矩陣MW的行方向上,權重W11~W1M、權重W21~W2M、...、權重WN1~WNM依序排列。透過重新定序,權重W11~WNM的權重讀出順序ORD不同於權重矩陣MW中的權重W11~WNM的排列順序。舉例來說,控制電路120可能基於權重讀出順序ORD先讀出權重W11~W1M,接著讀出權重W31~W3M,隨後讀出權重W21~W2M。
In this embodiment, during the process of writing the weights W 11 ~W NM into the
在本實施例中,運算電路130耦接於控制電路120。運算電路130基於權重讀出順序ORD來接收權重W11~WNM。因此,在行方向上,運算電路130所接收到的權重W11~WNM的列順序不同於權重矩陣MW中的權重W11~WNM的列順序。運算電路130還接收輸入資料矩陣MI,並對權重W11~WNM以及輸入資料矩陣MI進行矩陣運算以產生運算矩陣MC。在本實施例中,輸入資料矩陣MI例如是具有M個列以及1個行的一維矩陣(本發明並不以此為限)。因此,輸入資料矩陣MI包括輸入元素值IN1~INM。運算電路130對權重W11~WNM以及輸入資料矩陣MI進行矩陣乘法運算以產生運算矩陣MC。因此,運算矩陣MC是具有N個列以及1
個行的一維矩陣(本發明並不以此為限)。運算矩陣MC包括運算元素值E1~EN。
In this embodiment, the
控制電路120對運算矩陣MC進行維度轉換(reshape)以產生輸出矩陣MO。控制電路120將輸出矩陣MO寫入至儲存單元110。控制電路120會增加運算矩陣MC的維度以產生輸出矩陣MO。在本實施例中,控制電路120會將運算矩陣MC的維度從一維轉換為二維,從而產生輸出矩陣MO。控制電路120例如依序讀出運算元素值E1~EN,並將運算元素值E1~EN優先以列方式依序寫入至輸出矩陣MO。因此,矩陣MO包括運算元素值E11~ETS。應能理解的是,運算元素值E11等於E1。運算元素值ETS等於EN。
The
在此值得一提的是,控制電路120依據輸出矩陣MO的矩陣形狀來對權重矩陣MW中的權重W11~WNM的排列順序進行重新定序以確定出權重W11~WNM的權重讀出順序ORD。運算電路130基於權重讀出順序ORD來對權重W11~WNM以及輸入資料矩陣MI進行矩陣運算以產生運算矩陣MC。應注意的是,權重讀出順序ORD改變了運算矩陣MC的運算元素值E1~EN的排列順序。運算元素值E1~EN的排列順序有助於在進行維度轉換時就實現了轉置效果。如此一來,矩陣運算裝置100並不需要利用額外的轉置工具來對運算矩陣MC或輸出矩陣MO進行轉置運算。矩陣運算裝置100的運行成本並不會被增加。
It is worth mentioning here that the
在本實施例中,控制電路120可以是由邏輯電路、記憶體控制器、輸入/輸出緩衝器(I/O buffer)或中央處理單元(CPU)
來實施。在本實施例中,運算電路130可適用於類神經網路(neural network,NN)的矩陣運算。
In this embodiment, the
在一些實施例中,輸入資料矩陣MI可以是由外部裝置來提供。在一些實施例中,輸入資料矩陣MI可以是由儲存單元110來提供。
In some embodiments, the input data matrix MI may be provided by an external device. In some embodiments, the input data matrix MI may be provided by the
為了便於說明,權重矩陣MW以二維陣列來示例。輸入資料矩陣MI以一維陣列來示例。然本發明並不以此為限。在一些實施例中,權重矩陣MW可以是多列且單行的一維陣列。輸入資料矩陣MI以可以是二維陣列。 For ease of explanation, the weight matrix MW is exemplified as a two-dimensional array. The input data matrix MI is exemplified as a one-dimensional array. However, the present invention is not limited to this. In some embodiments, the weight matrix MW may be a one-dimensional array with multiple columns and a single row. The input data matrix MI can be a two-dimensional array.
請同時參考圖2以及圖3,圖3是依據本發明一實施例所繪示的矩陣運算裝置的示意圖。在本實施例中,權重矩陣MW包括多個權重列。第1權重列包括權重W11~W1M。第2權重列包括權重W21~W2M。第(T+1)權重列包括權重W(T+1)1~W(T+1)M。第(2T+1)權重列包括權重W(2T+1)1~W(2T+1)M。同理可推,第N權重列包括權重WN1~WNM。控制電路120依據輸出矩陣MO的行數以及列數以交錯(interleave)方式來確定出權重W11~WNM的權重讀出順序ORD。
Please refer to FIG. 2 and FIG. 3 at the same time. FIG. 3 is a schematic diagram of a matrix operation device according to an embodiment of the present invention. In this embodiment, the weight matrix MW includes multiple weight columns. The first weight column includes weights W 11 ~W 1M . The second weight column includes weights W 21 ~W 2M . The (T+1)th weight column includes weights W (T+1)1 ~W (T+1)M . The (2T+1)th weight column includes weights W (2T+1)1 ~W (2T+1)M . By the same token, the Nth weight column includes weights W N1 ~W NM . The
在本實施例中,輸出矩陣MO例如是具有T個列以及S個行的二維矩陣。控制電路120會將權重矩陣MW的第1權重列作為第1讀出列RO1,並將權重矩陣MW的第(nT+1)權重列作為第(n+1)讀出列RO(n+1)。n小於S。控制電路120將權重矩陣MW的第2權重列作為第(S+1)讀出列RO(S+1),並將權重矩陣MW的
第(nT+2)權重列作為第(S+n+1)讀出列(未示出)。因此,基於權重讀出順序ORD所產生的讀出矩陣MW’被形成。換言之,控制電路120依據權重讀出順序ORD將權重矩陣MW轉換為讀出矩陣MW’。第1讀出列RO1包括權重W11~W1M。第2讀出列RO2包括權重W(T+1)1~W(T+1)M(即,n=1)。第3讀出列RO3包括權重W(2T+1)1~W(2T+1)M(即,n=2)。第(S+1)讀出列RO(S+1)包括權重W21~W2M。
In this embodiment, the output matrix MO is, for example, a two-dimensional matrix having T columns and S rows. The
運算電路130基於權重讀出順序ORD所接收到的權重W11~WNM的排列等同於讀出矩陣MW’的態樣。運算電路130會對讀出矩陣MW’以及輸入資料矩陣MI進行乘法運算以產生運算矩陣MC。運算元素值E1會等於第1讀出列RO1的權重W11~W1M與輸入元素值IN1~INM的乘法累加(Multiply Accumulate)值。運算元素值E2會等於第2讀出列RO2的權重W21~W2M與輸入元素值IN1~INM的乘法累加值,依此類推。運算元素值E1、E2如分別如公式(1)、公式(2)所示
控制電路120接收運算矩陣MC,並將運算矩陣MC的維度從一維轉換二維以產生輸出矩陣MO。應注意的是,權重讀出順序ORD改變了運算矩陣MC的運算元素值E1~EN的排列順序。運算元素值E1~EN的排列順序有助於在進行維度轉換時就實現了轉置效果。
The
在一些實施例中,控制電路120會將讀出矩陣MW’儲存至儲存單元110。因此,在權重W11~WNM不被更新的情況下,控制電路120可讀取讀出矩陣MW’而不需執行重新定序的操作。在一些實施例中,讀出矩陣MW’以及權重矩陣MW分別被儲存在儲存單元110的不同區塊(segment)。在一些實施例中,當讀出矩陣MW’被儲存至儲存單元110時,讀出矩陣MW’會覆蓋權重矩陣MW。
In some embodiments, the
舉例來說明,請同時參考圖4A以及圖4B,圖4A是現行的矩陣運算的簡易範例示意圖。圖4B是依據本發明一實施例所繪示的矩陣運算的簡易範例示意圖。圖4A示出了輸出矩陣MO的產生方式。在現行的矩陣運算中,權重矩陣MW會與輸入資料矩陣MI進行乘法運算以產生運算矩陣MC。因此,運算矩陣MC的運算元素值依序為“37”、“50”、“18”、“36”。經過維度轉換後,輸出矩陣MO的運算元素值同樣依序為“37”、“50”、“18”、“36”。應注意的是,當輸出矩陣MO被用於作為如圖1所示的矩陣MB時,輸出矩陣MO必須透過轉置運算以形成轉置矩陣MT,從而使運算元素值的排列改為“37”、“18”、“50”、“36”。輸出矩陣MO的產生已經涉及輸入元素值的接收。輸入元素值是類神經網路運作時所接收到的變數。因此,已完成的輸出矩陣MO的轉置運算是額外的矩陣運算。在類神經網路的應用中,輸出矩陣MO的轉置運算必須在類神經網路運作時進行。因此,輸出矩陣MO的轉置運算會耗費運算成本。 For example, please refer to FIG. 4A and FIG. 4B at the same time. FIG. 4A is a schematic diagram of a simple example of a current matrix operation. FIG. 4B is a schematic diagram of a simple example of matrix operation according to an embodiment of the present invention. Figure 4A shows how the output matrix MO is generated. In the current matrix operation, the weight matrix MW is multiplied by the input data matrix MI to generate the operation matrix MC. Therefore, the operation element values of the operation matrix MC are "37", "50", "18", and "36" in order. After dimension conversion, the operation element values of the output matrix MO are also "37", "50", "18", and "36" in order. It should be noted that when the output matrix MO is used as the matrix MB as shown in Figure 1, the output matrix MO must be formed through a transposition operation to form the transpose matrix MT, so that the arrangement of the operation element values is changed to "37" , "18", "50", "36". The generation of the output matrix MO already involves the reception of the input element values. The input element values are the variables received by the neural network when it operates. Therefore, the completed transpose operation of the output matrix MO is an additional matrix operation. In the application of neural network, the transpose operation of the output matrix MO must be performed when the neural network is operating. Therefore, the transpose operation of the output matrix MO will consume computational costs.
圖4B示出了本實施例的輸出矩陣MO的產生方式。在本實施例中,權重矩陣MW先被重新定序以產生讀出矩陣MW’。應注意的是,在類神經網路的應用中,權重是參數而不是變數。因此,權重矩陣MW的重新定序可以在離線(offline)狀態下完成。權重矩陣MW的重新定序可以不用在類神經網路運作時進行。也就是說,讀出矩陣MW’的產生並不會增加在類神經網路運作時的運算成本及功耗。讀出矩陣MW’會與輸入資料矩陣MI進行乘法運算以產生運算矩陣MC。因此,運算矩陣MC的運算元素值依序為“37”、“18”、“50”、“36”。經過維度轉換後,輸出矩陣MO的運算元素值同樣依序為“37”、“18”、“50”、“36”。圖4B所示的輸出矩陣MO等於如圖4A所示的轉置矩陣MT。也就是說,本實施例能夠增加權重矩陣MW的重新定序即可實現如圖4A輸出矩陣MO的轉置運算的結果。 FIG. 4B shows how the output matrix MO is generated in this embodiment. In this embodiment, the weight matrix MW is first reordered to generate the readout matrix MW'. It should be noted that in neural network-like applications, weights are parameters rather than variables. Therefore, the reordering of the weight matrix MW can be completed offline. The reordering of the weight matrix MW does not need to be performed when the neural network is operating. In other words, the generation of the readout matrix MW' will not increase the computational cost and power consumption when operating a neural network. The readout matrix MW' will be multiplied by the input data matrix MI to generate the operation matrix MC. Therefore, the operation element values of the operation matrix MC are "37", "18", "50", and "36" in order. After dimension conversion, the operation element values of the output matrix MO are also "37", "18", "50", and "36" in order. The output matrix MO shown in Figure 4B is equal to the transposed matrix MT shown in Figure 4A. That is to say, this embodiment can increase the reordering of the weight matrix MW to achieve the result of the transpose operation of the output matrix MO as shown in Figure 4A.
請同時參考圖2、圖3以及圖5,圖5是依據本發明一實施例所繪示的運算電路的電路示意圖。在本實施例中,運算電路230包括乘積累加電路231(1)~231(N)。乘積累加電路231(1)~231(N)分別透過不同的通道耦接至控制電路120。乘積累加電路231(1)~231(N)分別透過不同的通道以接收權重矩陣MW的對應權重列。乘積累加電路231(1)透過通道CH(1)耦接至控制電路120。乘積累加電路231(2)透過通道CH(2)耦接至控制電路120。同理可推,乘積累加電路231(N)透過通道CH(N)耦接至控制電路120。
Please refer to FIG. 2 , FIG. 3 and FIG. 5 at the same time. FIG. 5 is a circuit schematic diagram of a computing circuit according to an embodiment of the present invention. In this embodiment, the
以本實施例為例,乘積累加電路231(1)透過通道CH(1) 接收對應權重列(即,第1讀出列RO1)。因此,乘積累加電路231(1)會透過通道CH(1)在依序接收權重W11~W1M,並且對權重W11~W1M以及輸入資料矩陣MI進行乘積累加運算(multiply-accumulate computing,MAC)以產生運算矩陣MC的運算元素值E1。乘積累加電路231(2)透過通道CH(2)接收對應權重列(即,第2讀出列RO2)。因此,乘積累加電路231(2)會透過通道CH(2)在依序接收權重W(T+1)1~W(T+1)M,並且對權重W(T+1)1~W(T+1)M以及輸入資料矩陣MI進行乘積累加運算以產生運算矩陣MC的運算元素值E2。同理,乘積累加電路231(N)會透過通道CH(N)在依序接收第N讀出列RON的權重WN1~WNM,並且對權重WN1~WNM以及輸入資料矩陣MI進行乘積累加運算以產生運算矩陣MC的運算元素值EN。 Taking this embodiment as an example, the multiply-accumulate circuit 231(1) receives the corresponding weight sequence (ie, the first readout column RO1) through the channel CH(1). Therefore, the multiply-accumulate circuit 231(1) will receive the weights W 11 ~W 1M in sequence through the channel CH(1), and perform multiply-accumulate computing on the weights W 11 ~W 1M and the input data matrix MI. MAC) to generate the operation element value E 1 of the operation matrix MC. The multiply-accumulate circuit 231(2) receives the corresponding weight sequence (ie, the second readout column RO2) through the channel CH(2). Therefore, the multiply-accumulate circuit 231(2) will receive the weights W (T+1)1 ~W (T+1)M in sequence through the channel CH(2), and calculate the weights W (T+1)1 ~W ( T+1)M and the input data matrix MI perform a multiplication and accumulation operation to generate the operation element value E 2 of the operation matrix MC. In the same way, the multiply-accumulate circuit 231(N) will sequentially receive the weights W N1 ~W NM of the Nth read column RON through the channel CH(N), and multiply the weights W N1 ~W NM and the input data matrix MI The accumulation operation is performed to generate the operation element value EN of the operation matrix MC.
以乘積累加電路231(1)為例,乘積累加電路231(1)包括乘法器MU、暫存器RG以及加法器AD。暫存器RG在第一時間儲存運算元素值E1。此時,運算元素值E1可以是初始值(例如是“0”)。乘法器MU耦接於通道CH(1)以及輸入資料矩陣MI。乘法器MU在第一時間接收權重W11以及輸入資料矩陣MI中的輸入資料IN1,並對權重W11以及輸入資料IN1進行乘法運算以產生乘積值MV。加法器AD在第一時間接收儲存於暫存器RG的運算元素值E1以及來自於乘法器MU的乘積值MV。加法器AD對運算元素值E1以及乘積值MV進行加法運算以產生新的運算元素值E1,並將新的運算元素值E1儲存至及暫存器RG。在第二時間,乘法器MU接收權重W12以及輸入資料矩陣MI中的輸入資料IN2,並 對權重W12以及輸入資料IN2進行乘法運算以產生新的乘積值MV。加法器AD接收新的乘積值MV以及在第一時間儲存於暫存器RG的運算元素值E1。加法器AD對運算元素值E1以及新的乘積值MV進行加法運算以產生新的運算元素值E1,依此類推。 Taking the multiply-accumulate circuit 231(1) as an example, the multiply-accumulate circuit 231(1) includes a multiplier MU, a register RG, and an adder AD. The register RG stores the operation element value E 1 at the first time. At this time, the operation element value E 1 may be an initial value (for example, "0"). Multiplier MU is coupled to channel CH(1) and input data matrix MI. The multiplier MU receives the weight W 11 and the input data IN 1 in the input data matrix MI at the first time, and multiplies the weight W 11 and the input data IN 1 to generate a product value MV. The adder AD receives at the first time the operation element value E 1 stored in the register RG and the product value MV from the multiplier MU. The adder AD performs an addition operation on the operation element value E 1 and the product value MV to generate a new operation element value E 1 , and stores the new operation element value E 1 into the register RG. At the second time, the multiplier MU receives the weight W 12 and the input data IN 2 in the input data matrix MI, and multiplies the weight W 12 and the input data IN 2 to generate a new product value MV. The adder AD receives the new product value MV and the operation element value E 1 stored in the register RG at the first time. The adder AD adds the operation element value E 1 and the new product value MV to generate a new operation element value E 1 , and so on.
在本實施例中,乘積累加電路231(2)~231(N)的電路配置相似於乘積累加電路231(1)的電路配置,故不在此重述。 In this embodiment, the circuit configuration of the multiply-accumulate circuits 231(2)~231(N) is similar to the circuit configuration of the multiply-accumulate circuit 231(1), so it will not be repeated here.
請同時參考圖2以及圖6,圖6是依據本發明一實施例所繪示的操作方法的示意圖。操作方法S100適用於矩陣運算裝置100。操作方法S100包括步驟S110~S130。在步驟S110中,控制電路120依據輸出矩陣MO的矩陣形狀來對儲存單元110的權重矩陣MW中的權重W11~WNM的排列順序進行重新定序以確定出權重W11~WNM的權重讀出順序ORD。
Please refer to FIG. 2 and FIG. 6 at the same time. FIG. 6 is a schematic diagram of an operating method according to an embodiment of the present invention. The operation method S100 is applicable to the
在步驟S120中,運算電路130基於權重讀出順序ORD來接收權重W11~WNM,並對權重W11~WNM以及輸入資料矩陣MI進行矩陣運算以產生運算矩陣MC。
In step S120, the
在步驟S130中,控制電路120對運算矩陣MC進行維度轉換以產生輸出矩陣MO,並且將輸出矩陣MO寫入至儲存單元110。步驟S110~S130的實施細節已經在圖1至圖5的實施例清楚說明,故不在此重述。
In step S130 , the
綜上所述,矩陣運算裝置以及操作方法依據輸出矩陣的矩陣形狀來對權重矩陣中的多個權重的排列順序進行重新定序以確定出所述多個權重的權重讀出順序。運算電路基於權重讀出順 序來對所述多個權重以及輸入資料矩陣進行矩陣運算以產生運算矩陣。權重讀出順序改變了運算矩陣的元素排列順序。運算矩陣的元素排列順序有助於在進行維度轉換時就實現了轉置效果。因此,矩陣運算裝置並不需要利用額外的轉置工具來對矩陣進行轉置運算。本發明的矩陣運算裝置的運行成本並不會被增加。 To sum up, the matrix operation device and the operation method reorder the arrangement order of the plurality of weights in the weight matrix according to the matrix shape of the output matrix to determine the weight reading order of the plurality of weights. The arithmetic circuit reads out the order based on the weight. The matrix operation is performed on the plurality of weights and the input data matrix in order to generate an operation matrix. The weight reading order changes the order of elements of the operation matrix. The order in which the elements of the operation matrix are arranged helps achieve the transposition effect when performing dimension conversion. Therefore, the matrix operation device does not need to use additional transposition tools to perform transpose operations on the matrix. The operating cost of the matrix operation device of the present invention will not be increased.
雖然本發明已以實施例揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明的精神和範圍內,當可作些許的更動與潤飾,故本發明的保護範圍當視後附的申請專利範圍所界定者為準。 Although the present invention has been disclosed above through embodiments, they are not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some modifications and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the appended patent application scope.
100:矩陣運算裝置 100:Matrix operation device
110:儲存單元 110:Storage unit
120:控制電路 120:Control circuit
130:運算電路 130: Arithmetic circuit
E1~EN、E11~ETS:運算元素值 E 1 ~E N , E 11 ~E TS : operation element value
IN1~INM:輸入元素值 IN 1 ~IN M : Input element value
ORD:權重讀出順序 ORD: weight reading order
MC:運算矩陣 MC: operation matrix
MI:輸入資料矩陣 MI: input data matrix
MO:輸出矩陣 MO: output matrix
MW:權重矩陣 MW: weight matrix
W11~WNM:權重 W 11 ~W NM : weight
Claims (16)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW111139781A TWI814618B (en) | 2022-10-20 | 2022-10-20 | Matrix computing device and operation method thereof |
US18/076,407 US20240232286A9 (en) | 2022-10-20 | 2022-12-07 | Matrix computing device and operation method thereof |
CN202211566152.9A CN117917655A (en) | 2022-10-20 | 2022-12-07 | Matrix operation device and operation method thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW111139781A TWI814618B (en) | 2022-10-20 | 2022-10-20 | Matrix computing device and operation method thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI814618B true TWI814618B (en) | 2023-09-01 |
TW202418113A TW202418113A (en) | 2024-05-01 |
Family
ID=88966070
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW111139781A TWI814618B (en) | 2022-10-20 | 2022-10-20 | Matrix computing device and operation method thereof |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240232286A9 (en) |
CN (1) | CN117917655A (en) |
TW (1) | TWI814618B (en) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102567241A (en) * | 2010-12-27 | 2012-07-11 | 北京国睿中数科技股份有限公司 | Memory controller and memory access control method |
TW201346745A (en) * | 2011-12-30 | 2013-11-16 | Intel Corp | Transpose instruction |
TW201947587A (en) * | 2017-02-17 | 2019-12-16 | 美商谷歌有限責任公司 | Permuting in a matrix-vector processor |
CN111859273A (en) * | 2017-12-29 | 2020-10-30 | 华为技术有限公司 | Matrix multiplier |
TWI746126B (en) * | 2020-08-25 | 2021-11-11 | 創鑫智慧股份有限公司 | Matrix multiplication device and operation method thereof |
CN113850380A (en) * | 2021-09-26 | 2021-12-28 | 安徽寒武纪信息科技有限公司 | Data processing device, data processing method and related product |
CN114579929A (en) * | 2022-03-14 | 2022-06-03 | 海飞科(南京)信息技术有限公司 | Accelerator execution method and electronic device |
TW202232344A (en) * | 2019-12-23 | 2022-08-16 | 台灣積體電路製造股份有限公司 | System for processing data set |
-
2022
- 2022-10-20 TW TW111139781A patent/TWI814618B/en active
- 2022-12-07 US US18/076,407 patent/US20240232286A9/en active Pending
- 2022-12-07 CN CN202211566152.9A patent/CN117917655A/en active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102567241A (en) * | 2010-12-27 | 2012-07-11 | 北京国睿中数科技股份有限公司 | Memory controller and memory access control method |
TW201346745A (en) * | 2011-12-30 | 2013-11-16 | Intel Corp | Transpose instruction |
TW201947587A (en) * | 2017-02-17 | 2019-12-16 | 美商谷歌有限責任公司 | Permuting in a matrix-vector processor |
CN111859273A (en) * | 2017-12-29 | 2020-10-30 | 华为技术有限公司 | Matrix multiplier |
TW202232344A (en) * | 2019-12-23 | 2022-08-16 | 台灣積體電路製造股份有限公司 | System for processing data set |
TWI746126B (en) * | 2020-08-25 | 2021-11-11 | 創鑫智慧股份有限公司 | Matrix multiplication device and operation method thereof |
CN113850380A (en) * | 2021-09-26 | 2021-12-28 | 安徽寒武纪信息科技有限公司 | Data processing device, data processing method and related product |
CN114579929A (en) * | 2022-03-14 | 2022-06-03 | 海飞科(南京)信息技术有限公司 | Accelerator execution method and electronic device |
Also Published As
Publication number | Publication date |
---|---|
TW202418113A (en) | 2024-05-01 |
US20240232286A9 (en) | 2024-07-11 |
CN117917655A (en) | 2024-04-23 |
US20240134931A1 (en) | 2024-04-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Haj-Ali et al. | Efficient algorithms for in-memory fixed point multiplication using magic | |
EP3726399A1 (en) | Matrix multiplier | |
EP3555814B1 (en) | Performing average pooling in hardware | |
US20220188604A1 (en) | Method and Apparatus for Performing a Neural Network Operation | |
CN111796797B (en) | Method and device for realizing loop polynomial multiplication calculation acceleration by using AI accelerator | |
Gebali et al. | Efficient Scalable Serial Multiplier Over GF ($\textbf {2}^{\boldsymbol {m}} $) Based on Trinomial | |
JP7435602B2 (en) | Computing equipment and computing systems | |
Wang et al. | Solving large systems of linear equations over GF (2) on FPGAs | |
TW202203053A (en) | Memory circuit and operating method thereof | |
CN110673824A (en) | Matrix vector multiplication circuit and circular neural network hardware accelerator | |
TWI814618B (en) | Matrix computing device and operation method thereof | |
WO2019206162A1 (en) | Computing device and computing method | |
CN112784951A (en) | Winograd convolution operation method and related product | |
Katti et al. | Low complexity multiplication in a finite field using ring representation | |
Singh et al. | XCRYPT: Accelerating Lattice-Based Cryptography With Memristor Crossbar Arrays | |
JP5157484B2 (en) | Matrix operation coprocessor | |
CN103765493B (en) | Digital square computer implemented method and apparatus | |
US11435981B2 (en) | Arithmetic circuit, and neural processing unit and electronic apparatus including the same | |
CN112966729A (en) | Data processing method and device, computer equipment and storage medium | |
CN110889080B (en) | Multiply-accumulate operation device, multiply-accumulate operation method and system | |
CN114706557B (en) | ASIC chip and implementation method and device of Montgomery modular multiplication | |
US20220012222A1 (en) | Indexing Elements in a Source Array | |
JP7023149B2 (en) | Semiconductor device | |
TW202244794A (en) | Method and non-transitory computer readable medium for compute-in-memory macro arrangement, and electronic device applying the same | |
WO2021212972A1 (en) | Operation method, processor, and related product |