JP7136343B2 - Data processing system, method and program - Google Patents

Description

The present invention relates generally to data processing systems, methods, and programs based on one-dimensional array architectures and, more particularly, to continuous matrix processing, including matrix transpose calculations.

Machine learning has become very popular in recent years due to its high performance in many research areas. As more and more applications for machine learning are developed, the computational complexity increases significantly. Efficient data processing is therefore very important. Increasing the parallelism of vector processing to improve computational efficiency is highly beneficial and therefore preferred.

In vector processing, the main concept is to process data in vector patterns. Vector lanes containing an arithmetic logic unit (ALU) in each vector lane are used to process the matrix data. Each vector lane must contain local memory to store computational data. Additionally, direct memory access (DMA) can be used to transfer data between on-chip local memory and off-chip external memory. Finally, for the overall control logic, a central control unit is preferably used in the system. Based on the above concepts, several vector processor designs are described in US Pat.

In machine learning, one of the most common computations is matrix computation. Generalized Matrix Multiplication (GEMM) is used in both forward and backward propagation. When performing the GEMM function, all source data must be retrieved from memory, and two-dimensional processing elements are used to fully exploit the data fetched from each element. However, a drawback of 2D arrays is that if the actual matrix size of the computation is much smaller than the supported 2D array size, the computational resources of unused processing elements are wasted. Alternatively, one-dimensional arrays can be used as they are more flexible.

U.S. Patent No. 005600843

"Application-Specific Soft-Core Vectors for Advanced Driver Assistance Systems" by Stephan Nolting et al., published September 2017 at the International Conference on Field Programmable Logic and Applications. Processor (Application-Specific Soft-Core Vector Processor for Advanced Driver Assistance Systems)” “Fully Pipelined Soft Vector Processors as CPU Accelerators” by Yeyong Pang et al. Fully Pipelined Soft Vector Processor as a CPU Accelerator)

A first problem with the prior art is that matrix transposition is not generally supported in vector processors. In machine learning, matrix transpose is used in several cases, such as backpropagation of fully connected layers. The reference does not disclose how to efficiently transpose the matrix within the processor. Therefore, if the particular instruction for matrix transposition is not supported by the vector processor, the transposition must be performed outside the vector processor. To do this, the matrix to be transposed must first be transferred from local memory within the vector processor to external memory outside the processor. After the transposed result is prepared in the external memory, the transposed matrix needs to be transferred from the external memory outside the vector processor to the vector processor, which wastes a lot of time in data transfer.

The second problem is that many data transfers are mainly composed of two types. One is the transfer between the local memory itself and the other is the transfer between the local memory and the external memory. Data transfers between local memory and external memory usually have bubble cycles because external memory is not always available in current vector processing units. Also, when transferring data between local memory and external memory, even if there are multiple bubble cycles, the requests are presented one after another in succession, making it impossible to process other transfer requests. Overall system throughput performance is degraded.

One exemplary object of the present invention is to provide a matrix transposing device that can solve the first problem (identified above) that matrix transposing is not supported in conventional vector processing engines. .

Another exemplary object of the present invention is to provide a parallel system that can solve the second problem (identified above) of not being able to accept multiple requests for a single local memory simultaneously. .

A first aspect of the present disclosure is a central processing unit; a vector processing unit electronically coupled to the central processing unit and configured to perform operations based on instructions received from the central processing unit; an instruction memory unit electronically connected to the central processing unit and configured to store instructions; an external memory unit electronically connected to the central processing unit and configured to store one-dimensional systolic data; a second local memory unit electronically coupled to said vector processing device and configured to store matrix data; said first local memory unit; a second local memory unit, said instruction memory unit, and a direct memory access unit electronically coupled to said external memory unit and configured to access data in said external memory unit. and data is transferred through the direct memory access unit with timing based on a predetermined selection priority.

A second aspect of the disclosure provides a method for a data processing system. The data processing system includes a central processing unit, a vector processing unit electronically connected to the central processing unit, an instruction memory unit electronically connected to the central processing unit, an external memory unit, and a a first local memory unit electronically connected to a processing device; a second local memory unit electronically connected to said vector processing device; said first local memory unit; said second local memory; a direct memory access unit electronically connected to said instruction memory unit and said external memory unit. The method includes: performing operations based on instructions received from the central processing unit by the vector processing unit; storing instructions by the instruction memory unit; storing instructions by the first local memory unit; storing one-dimensional systolic data; storing matrix data by the second local memory unit; and accessing data in the external memory unit by the direct memory access unit. . Data is transferred through the direct memory access unit with timing based on predetermined selection priorities.

A third aspect of the present disclosure provides a program for a data processing system. The data processing system includes a central processing unit, a vector processing unit electronically connected to the central processing unit, an instruction memory unit electronically connected to the central processing unit, an external memory unit, and a a first local memory unit electronically connected to a processing device; a second local memory unit electronically connected to said vector processing device; said first local memory unit; said second local memory; a direct memory access unit electronically connected to said instruction memory unit and said external memory unit, said program instructing said vector processing unit to act on instructions received from said central processing unit. , causing the instruction memory unit to store instructions, causing the first local memory unit to store one-dimensional systolic data, causing the second local memory unit to store matrix data, and causing the direct A memory access unit is caused to access data in the external memory unit. Data is transferred through the direct memory access unit with timing based on predetermined selection priorities.

One advantage of the present invention is that matrix transposition can be performed in local memory. The reason for this effect is the ability to create specific instructions for matrix transposition.

When the vector processor receives this command, it begins transferring data from one local memory to the other. During the transfer, each data item from the source matrix is entered in the destination matrix in a transposed manner, as the mapped addresses for the source and destination memories are calculated.

A second advantage is that data can be transferred between the two local memories and the external memory within the same period.

The reason for this effect is that a priority request selection scheme is implemented so that a low priority data transfer can be performed during the bubble cycle of a high priority data transfer.

1 is a block diagram showing the structure of a first exemplary embodiment of the present invention; FIG. Fig. 3 is a block diagram showing the structure of the local memory of the first exemplary embodiment; Fig. 3 is a block diagram showing the structure of the local memory of the first exemplary embodiment; Fig. 3 is a block diagram showing the structure of the processing element of the first exemplary embodiment; Fig. 4 is a block diagram showing the structure of matrix transfer of the first exemplary embodiment; Figure 3 is a block diagram illustrating data mapping of local memory for the first exemplary embodiment; Figure 3 is a block diagram illustrating data mapping of local memory for the first exemplary embodiment; FIG. 4 is a block diagram illustrating priorities of different memory access requests for the local memory of the first exemplary embodiment; FIG. 4 is a block diagram illustrating priorities of different memory access requests to the local memory of the first exemplary embodiment; FIG. 2 is a flow diagram showing the procedure for two consecutive GEMM computations without matrix transposition according to conventional methods; FIG. 4 is a flow diagram showing the procedure for two consecutive GEMM computations without matrix transposition according to the method of the first exemplary embodiment; FIG. 2 is a flow diagram showing the procedure for two successive GEMM computations with matrix transposition according to conventional methods; Fig. 2 is a flow diagram showing the procedure for two successive GEMM computations with matrix transposition according to the method of the first exemplary embodiment; Fig. 2 is a block diagram showing the mechanism of two requests operating in the same period using the priority selection of the first exemplary embodiment; FIG. 4 is a block diagram showing the structure of another exemplary embodiment of the present invention;

(Description of configuration)
First, a first exemplary embodiment of the present invention will be detailed below with reference to the accompanying drawings.

Referring to FIG. 1, in a first exemplary embodiment of the present invention, data processing system 100 includes central processing unit (CP) 110, vector processing (VP) engine 120, DMA 130, matrix transfer device 140, data storage a first local memory 150 for data storage, a second local memory 160 for data storage, an instruction memory 180 for instruction storage, and an external memory 170 .

CP 110, which may be a MIPS processor, or similar architecture processor, supports supporting basic instructions such as arithmetic calculations, and uses general purpose registers within central processing unit 110 to/from external memory 170. Store/load from Central processing unit 110 controls first and second local memories 150 and 160 to fetch computation data from external memory 170 and prepare all instructions stored in instruction memory 180 . Central processing unit 110 then transmits vector instructions and matrix data to vector processing unit 120 . The vector processor 120 receives instructions and begins computation. The calculations of vector processor 120 are based on one-dimensional systolic patterns. One input is fetched from the first local memory 150 and the other input is fetched from the second local memory 160 . After calculation, the results are stored in the second local memory 160 . There is a path between the first local memory 150 and the second local memory 160 to transfer data between the first local memory 150 and the second local memory 160 via the matrix transfer device 140 be able to. Transfers between the first and second local memories 150, 160 can be normal transfers or transposed transfers. When the computation is finished, the results in second local memory 160 may be transferred to external memory 170 via direct memory access 130 .

A detailed architecture within the vector processor 120 is shown in FIG. Each of the plurality of processing units 121 is connected to each other. A connection exists between the central processing unit 110 and the first processing element 121 that is used to transfer data values and instruction information. Between adjacent processing elements 121 are connecting channels that are used to broadcast information to each processing element 121 . Internal to each processing element 121, a digital signal processor (DSP) can be used to efficiently compute the multiplications and additions to achieve lower power and higher frequencies. In addition, there are dedicated registers 125 in each processing element 121 to store intermediate results. Each processing element 121 has one dedicated register 125 .

The architectural details of local memory 150 are shown in FIG. There are 16 two-port RAM banks 151 and one first data select unit 153 . A 2-port RAM 151 is used to store data. The first data selection unit 153 selects data transfer with the external memory 170 or the second local memory 160 . There are also outputs as systolic data inputs for the vector processor 120 . Note that the number of 2-port RAM banks of the first local memory 150 in this exemplary embodiment is 16, but could be 8, 32, and so on.

Details of the architecture of the second local memory 160 are shown in FIG. There is a two port RAM bank 161, a second data select unit 162 and a data ring 163. The number of RAM banks 161 is equal to the number of processing elements 121 and one processing element 121 corresponds to one RAM bank 161 .

The architectural details of matrix transfer device 140 are shown in FIG. Inside the matrix transfer device 140 are an address generator 141 and a bank number generator 142 . Since the first local memory 150 and the second local memory 160 have different configurations, regardless of the type of data transferred from one local memory to the other local memory, the bank of the destination bank and the address of each bank are , may differ from the bank and address of the source bank. Therefore, address generator 141 and bank number generator 142 are used.

[Explanation of operation]
The general operation of the exemplary embodiment will now be described in detail with reference to the flowcharts of FIGS. 10-13.

First, a direct memory access unit (DMA) transfers instructions from external memory 170 to instruction memory 180 . For each application, the application can be compiled into assembly code. Therefore, assembly code is created for each application and stored in instruction memory 180 .

After that, central processing unit 110 fetches instructions from instruction memory 180 one by one. Initial data is stored in the first local memory 150 and the second local memory 160 before starting the computation. When performing a generalized matrix multiplication (GEMM) function, the left matrix is stored in the first local memory 150 and the right matrix is stored in the second local memory 160 . For each matrix M*K, the data mapping in the first local memory 150 is explained with reference to FIG. If there are 16 banks of RAM used in the first local memory 150, then D1[0,0] (D1[m,k] represents the mth row and kth column element of the matrix ) is stored at address 0 of bank 0 of the first local memory 150 to store the matrix. For example, data D1[0,1] is stored at address 0 of bank 1 of first local memory 150, and data D1[0,15] is stored at address 0 of bank 15 of first local memory 150, for example. stored in Starting at D1[0,16], data is stored at address 1 of bank 0 of the first local memory 150 .

Therefore, the address generator 141 and corresponding bank are calculated by the following equations.
ADDR=(m*K+k)/16
BANK=(m*K+k)%16

For data mapping of the second local memory 160 , the number of RAM banks 161 is equal to the number of processing elements 121 since each processing element 121 corresponds to one RAM bank 161 . In an example where there are 256 processing elements 121, a data mapping method is given with reference to FIG. D2[0,0] is stored at address 0 of bank 0 of the second local memory 160, D2[0,1] is stored at address 0 of bank 1 of the second local memory 160, and so on. is. For D2[1,0], the data is stored at address 1 of bank 0 of the second local memory 160 . For D2[1,1], the data is stored at address 1 of bank 1 of the second local memory 160 . For each matrix K*N, if N is less than the number of PEs 121, unused columns are filled with zeros. If N is greater than the number of PEs 121 , the matrix is cut column by column according to the number of PEs 121 and then mapped to the second local memory 160 . Thus, for element D2[k,n] (where k represents row and n represents column), the address generator and corresponding bank are given by the following equations.
ADDR=k
BANK=n

To store the matrix in the second local memory 160, a data ring 163 is used to pass data through all the 2-port RAM banks and write/read data to/from the corresponding 2-port RAM banks. For example, if a cache line is 64 bytes and uses 32-bit single precision floating point, there are 16 data items in a cache line. Each data item must be stored in a corresponding 2-port RAM bank 161 .

Ｄ１［０，０］は、ローカルメモリ１５０から読み取られ、プロセッシングエレメント１２１に転送される。Ｄ２［０，０］は、第２のローカルメモリ１６０から読み取られ、プロセッシングエレメント１２１のための他の入力オペランドとなる。計算後、Ｄ１［０，０］とＤ２［０，０］の乗算は専用レジスタに格納される。Ｄ１［０，０］は、シストリック方式で第２のプロセッシングエレメント１２１に転送される。Ｄ２［０，１］は、第２のローカルメモリ１６０から読み取られる。Ｄ１［０，０］とＤ２［０，１］は、乗算の２つのオペランドである。Ｄ１［０，０］＊Ｄ２［０，１］の結果は、専用レジスタ１２５に格納される。後のプロセッシングエレメント１２１については、Ｄ１［０，０］は常にシストリックの方法で転送される。最後に、Ｄ１［０，０］は、すべてのプロセッシングエレメント１２１を介して転送される。実際、各プロセッシングエレメント１２１のデータフローは完全に同じである。したがって、１つのプロセッシングエレメント１２１のデータフローを以下に簡単に説明する。 After storing the left and right matrices in the first local memory 150 and the second local memory 160, the computation is performed in a one-dimensional systolic manner. For GEMM computation, if the left matrix is D1(M,K) and the right matrix is D2(K,N), the multiplication result is RES(M,N). For each element of RES[m,n], the following computations are performed.

D1[0,0] is read from local memory 150 and transferred to processing element 121 . D2[0,0] is read from the second local memory 160 and becomes another input operand for the processing element 121; After computation, the multiplication of D1[0,0] and D2[0,0] is stored in a dedicated register. D1[0,0] is transferred to the second processing element 121 in a systolic fashion. D2[0,1] is read from the second local memory 160; D1[0,0] and D2[0,1] are the two operands of the multiplication. The result of D1[0,0]*D2[0,1] is stored in special purpose register 125. FIG. For later processing elements 121, D1[0,0] is always transferred in a systolic manner. Finally, D1[0,0] is transferred through all processing elements 121 . In fact, the data flow for each processing element 121 is exactly the same. Therefore, the data flow for one processing element 121 is briefly described below.

After D1[0,0] has been transferred through all PEs 121 , the next data item is read from the first local memory 150 . The left matrix is read in row-major order. So, for example, D1[0,1] is read from the first local memory 150 and sent to the first processing element 121, and D2[1,0] is sent to the second local memory 121 in the first processing element 121. Read from memory 160 . Then D1[0,1]*D2[1,0] is calculated. The previous result of D1[0,0]*D2[0,0] is read from the special purpose register 125 and added to the result of D1[0,1]*D2[1,0]. The sum is stored again in dedicated register 125 . By repeating this, the first element RES[0,0] of the first processing element 121 can be acquired. Similarly, all elements within the first processing element 121 can be computed. After computing the elements, the results are sent to the two-port RAM bank 161 in the second local memory 160 .

Data transfer between the first local memory 150 and the second local memory 160 will be described below. There are four cases. The first case is a normal transfer from first local memory 150 to second local memory 160 . Assuming that the matrix size stored in the second local memory 160 is K×N, N processing elements 121 are used and each RAM bank is K deep. As noted above, when N is less than the number of processing elements 121 (eg, 256), unused columns are filled with zeros. Thus, D2[0,0] in second local memory 160 maps to the first element of the first RAM bank in local memory 150 . D2[0,1] in the second local memory 160 is mapped to the first element of the second RAM in the first local memory 150; However, for D2[0,n] in the second local memory 160, if n is greater than the number of banks supported by the first local memory 150, the first local memory 150 and the second local memory The addresses and banks within 160 are different. For other element address and bank mappings, the calculation method is shown in the following equations, where DST_ADDR and DST_BANK are the destination address and bank respectively. In this case the destination address and bank are in the first local memory 150 . SRC_ADDR and SRC_BANK represent the source address and source bank, where the source address and source bank are in the second local memory 160 . SRC_NUM_BANK is the number of 2-port RAM banks 161 in the second local memory 160 and DST_NUM_BANK is the number of 2-port RAM banks 151 supported in the first local memory 150 .
DST_ADDR=(SRC_ADDR*SRC_NUM_BANK+SRC_BANK)/DST_NUM_BANK
DST_BANK=(SRC_ADDR*SRC_NUM_BANK+SRC_BANK)% DST_NUM_BANK

In a practical situation, on each clock cycle, 512 bits of data are transferred assuming the cache line is 512 bits. For 32-bit single precision floating point, one data item is 32 bits, so 16 data items are transferred in one clock cycle. Assuming there are 256 processing elements 121, for data transfer between the first local memory 150 and the second local memory 160, in the first clock cycle, D2[0,0], D2[0 ,1] . . . D2[0,15] are retrieved from the RAM 161 of each processing element 121 and transferred to the first local memory 150 via the data ring. D2[0,0], D2[0,1] . Local memory 150 can be written. In the second clock cycle, D2[0,16], D2[0,17], D2[0,18]... D2[0,31] are fetched from the RAM bank 161 of the corresponding processing element 121 It is transferred to the first local memory 150 via the data ring 163 . D2[0,16], D2[0,17] . can be written. Similarly, in each clock cycle, 16 data items are read from the RAM bank 161 of the corresponding processing element 121, then transferred via the data ring 163, and finally stored in the first local memory 150. be done.

The second case is a normal transfer from first local memory 150 to second local memory 160 . Similarly, the address and bank generators are the same as above, the only difference being that the source will be the first local memory 150 and the destination will be the second local memory 160 .

A third case is that the transposed matrix is transferred from the first local memory 150 to the second local memory 160 . Assuming the matrix [N,M] is stored in the first local memory 150 , this matrix is transposed to size [M,N] and stored in the second local memory 160 . For example, D1[0,0] of the first local memory 150 is mapped to address 0 of bank 0 of the second local memory 160 . D1[0,1] of the second local memory 160 is mapped to address 1 of bank 0 of the second local memory 160 . The following formula shows the calculation method for the mapping of addresses of other elements and banks.
DST_ADDR=(SRC_ADDR*SRC_NUM_BANK+SRC_BANK)% DST_NUM_BANK
DST_BANK=(SRC_ADDR*SRC_NUM_BANK+SRC_BANK)/DST_NUM_BANK

Transfers are still based on 16x16 blocks. However, this 16×16 block resides in the second local memory 160 . Therefore, to generate a 16×16 block on the second local memory 160 side every 16 clock cycles, we need D2[0,0], D2[1,1], D2 of the first local memory 150 [2,2]...D2[15,15] need to find corresponding elements. The corresponding address of D2[0,0] is address 0 of bank 0, the corresponding address of D2[1,1] is address (N+1)/16 of bank 1, and the corresponding address of D2[2,2] is The address to be used is the bank 2 address (N.times.2+2)/16. Similarly, the corresponding location for D2[15,15] is bank 15 address (N×15+15)/16. Therefore, 16 data items can be fetched in one clock cycle. At the second clock cycle, the corresponding elements of D2[1,0], D2[2,1], D2[3,2] . The corresponding address of D2[1,0] is bank 0 address N/16 and the corresponding address of D2[2,1] is bank 1 address (N×2+1)/16. Similarly, the corresponding location in D2[0,15] is bank 15, address 0. These mapping methods allow the first local memory 150 address of the 16×16 block of local memory 160 to be known. Each clock cycle, 16 elements fetched from the first local memory 150 are transposed and stored in the second local memory 160 . For example, in the first cycle, D2[0,0], D2[1,1], D2[2,2] . . . D2[15,15] are diagonal, so the transposed result is the same. For the second local memory 160, the address for bank 0 is 0, the address for bank 1 is 1, and so on. D2[1,0], D2[2,1], D2[3,2] . . . D2[0,15] are fetched from the first local memory 150 in the second clock cycle. For D2[1,0], the transposed result must be stored in bank 1 at address 0, D2[2,1] must be stored in bank 2 at address 1, D2[3 , 2] must be stored in bank 3 at address 2. Similarly, all data can be stored in 16 banks of the second local memory 160 in one clock cycle.

The address generator of the first local memory 150 can be expressed as the following equation, where CNT is the count of each 16 clock cycles of a 16×16 block and SRC_BANK is the count of the first local memory 150 It means which bank.
SRC_ADDR_IN_B16=(N*(SRC_BANK+CNT)+SRC_BANK)/16 if SRC_BANK+CNT<=15
SRC_ADDR_IN_B16=(N*(SRC_BANK+CNT-16)+SRC_BANK)/16 if SRC_BANK+CNT>15

The base address of a 16x16 block consists of two parts, one part corresponds to the vertical movement of the 16x16 block marked as delta_v, and the other part is the 16x16 marked as delta_h. Corresponds to horizontal movement of the block. B16_vertical_num represents the vertical address of the 16×16 block in the second local memory 160; Here a vertical scan of 16×16 blocks in the second local memory 160 is used.
B16_vertical_num=(B16_vertical_num<M/16)? B16_vertical_num+1:0
Delta_v=(B16_vertical_num<M/16)? (Delta_v+N): 0
Delta_h=(B16_vertical_num==0)? (Delta_h+1): Delta_h
SRC_B16_ADDR = Delta_v + Delta_h

Therefore, the final address of the 16 banks of the first local memory 150 can be obtained by the following formula.
SRC_ADDR = SRC_ADDR_IN_B16 + SRC_B16_ADDR

The address generator for the first local memory 150 is given above. Next, the address and bank generator of the second local memory 160 will be described. Since scanning the source matrix is vertical scanning in the second local memory 160 , scanning the transposed matrix is horizontal scanning in the second local memory 160 . Therefore, the base address and base bank can be calculated as follows.
DST_ADDR_IN_B16=(DST_BANK%16-CNT) if DST_BANK%16-CNT>=0
DST_ADDR_IN_B16 = (DST_BANK%16-CNT+16) if DST_BANK%16-CNT<0
DST_B16_ADDR=(DST_B16_BANK<N/16)? DST_B16_ADDR: (DST_B16_ADDR+16)
DST_ADDR = DST_ADDR_IN_B16 + DST_B16_ADDR

A fourth case is that the transposed matrix is transferred from the second local memory 160 to the first local memory 150 . Assuming the matrix [M,N] is stored in the second local memory 160 , this matrix needs to be transposed to [N,M] and stored in the first local memory 150 . For example, D2[0,0] of the second local memory 160 continues to map to address 0 of bank 0 of the first local memory 150 . D2[0,1] of the second local memory 160 is mapped to address 1 of bank 0 of the first local memory 150 . The following formula shows the calculation method for the mapping of addresses of other elements and banks.
DST_ADDR=(SRC_BANK*M+SRC_ADDR)/DST_NUM_BANK
DST_BANK=(SRC_BANK*M+SRC_ADDR) %DST_NUM_BANK

If the data in D2[0,0], D2[0,1] . stored in a bank. Therefore, we cannot store these 16 data elements in the first local memory 150 in one clock cycle. To avoid this problem, a cyclic data mapping method is used. The transfer is based on 16x16 blocks, which means that after accessing one 16x16 block of data, the 16x16 block is shifted to the next 16x16 block. The top left 16x16 block is transferred first and takes 16 clock cycles. D2[0,0], D2[1,1], D2[2,2] . These elements are stored in different banks of the first local memory 150 . D2[0,0] is stored in the first RAM of the first local memory 150, D2[1,1] is stored in the second RAM of the local memory, and so on. Therefore, we can store these 16 data items in the first local memory 150 in one clock cycle. In the second clock cycle, D2[0,1], D2[1,2], D2[2,3] . Read from RAM bank 161 . These elements are also stored in different banks of the first local memory 150 . Therefore, we can store these 16 data in the first local memory 150 in one clock cycle. The address generator is the same as in the third case, the only difference is that in the third case it is transferred from the first local memory 150 to the second local memory 160, whereas in this fourth case the second is transferred from the first local memory 160 to the first local memory 150 . Therefore, the source and destination addresses have been swapped.

Regarding the second local memory 160, transfers between the external memory 170 and the second local memory 160 and transfers between the first local memory 150 and the second local memory 160 are possible. The priority is determined by the data selection unit 162 if multiple transfer requests are received in the same clock cycle. A description of this operation is provided below with reference to the flow chart of FIG. The first priority is data transfer from external memory 170 to second local memory 160 . This is because the data must be retrieved from the external memory 170 if the data is valid. A second priority is the transfer from the second local memory 160 to the external memory 170 . The third priority is the transfer from the first local memory 150 to the second local memory 160 and the last priority is the transfer from the second local memory 160 to the first local memory 150. .

For the first local memory 150 a first data selection unit 153 is used. If multiple transfer requests are received in the same clock cycle, selection is made based on priority, as shown in FIG. The first priority is data transfer from external memory 170 to first local memory 150 . A second priority is the transfer from the first local memory 150 to the external memory 170 . The third priority is the transfer from the second local memory 160 to the first local memory 150 and the last priority is the transfer from the first local memory 150 to the second local memory 160. .

For consecutive GEMMs (eg, the first GEMM is A*B=C and the second GEMM is C*D=E). Description will be made below with reference to FIG. The first GEMM is A*B=C, with matrix A stored in the first local memory 150 and matrix B stored in the second local memory 160 . After storing matrix A in the first local memory 150 and matrix B in the local memory 160 , the computation is performed and the result is stored in the second local memory 160 . If the second GEMM is C*D=E, the matrix C needs to be stored in the first local memory 150 as a systolic one-dimensional input. To store the matrix C in the first local memory 150 , the result must first be sent to the external memory 170 and then transferred from the external memory to the first local memory 150 . Matrix D is then stored in the second local memory 160 and computation begins. Finally, the calculation result is obtained and transferred to the external memory 170 .

However, in our method we can transfer the matrix C directly from the second local memory 160 to the first local memory 150 . Therefore, we can omit the transfer of matrix C to external memory 170 . FIG. 11 shows the processing procedure. The computation of the first GEMM can be parallelized when storing the matrix D of the second GEMM.

Similarly, the second GEMM is C.I. T*D=E, where C. If T is the transposed matrix of C, in the conventional method, the procedure is shown in FIG. For computing the first GEMM, the procedure is the same as shown in FIG. After storing C in external memory, central processing unit 110 transposes matrix C in external memory 170 . The transposed matrix of C is then transferred to the first local memory 150, matrix D is transferred to the second local memory 160, and the second GEMM can begin operation.

However, the method of the present disclosure not only allows the matrix C to be directly transferred from the second local memory 160 to the first local memory 150, but also allows the transposition to be completed during the transfer. Therefore, the transfer of matrix C to external memory 170 can be performed with fewer operations. Furthermore, the transfer time to the central processing unit can be shortened. FIG. 13 shows the processing procedure.

In the example above, there are no concurrent requests to the same local memory (ie, first local memory 150 or second local memory 160). However, in some implementations these processes may occur. For example, after computing the first GEMM A*B=C, the result of matrix C needs to be transferred to external memory 170 . Meanwhile, the result also needs to be transferred to the first local memory 150 for the next GEMM, C*D=E. In this case, transfers between the second local memory 160 and the external memory 170 have higher priority. As shown in FIG. 14, data transfer from/to external memory 170 has several bubble cycles, and these bubble cycles are used to You can perform transfers between

[Explanation of effect]
Next, the effect of this exemplary embodiment will be described.

The exemplary embodiment is configured such that mapping addresses are provided periodically to the two local memories so that data can be transferred in a permuted manner via the data ring 163 inside the accelerator. , so there is no need to perform transposition on the host side.

Further, the exemplary embodiment is configured such that data transfer and computation use two different channels, allowing serial matrix computations to be performed in parallel. Furthermore, the data transfer of the previous matrix calculation can be performed concurrently with the calculation of the next matrix calculation.

In addition, multiple write/read requests can be performed in the same period per local memory, thus taking advantage of the bubble cycle of communication between local memory and external memory.

15, in another exemplary embodiment, data processing system 100 includes central processing unit (CP) 110, vector processing (VP) engine 120, DMA 130, first local memory 150 for data storage, It includes a second local memory 160 for data storage, an instruction memory 180 for instruction storage, and an external memory 170 .

The above program may be for partially executing the above functions. The above program may be a so-called difference file (difference program) combined with a program already recorded in the computer system to perform the above functions.

All or part of the functions of the data processing system described above may be performed by utilizing hardware such as ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays) and the like.

Additionally, features of the exemplary embodiments described above may be appropriately replaced with well-known features without departing from the scope of the invention. Moreover, the technical scope of the present invention is not limited to the exemplary embodiments described above, and various modifications can be made without departing from the scope of the present invention.

The present invention is applicable to data processing apparatus involving large amounts of vector or matrix computations, such as image or video processing platforms and deep learning platforms.

100 data processing system 110 central processing unit (CP)
120 vector processor (VP)
121 Processing Element (PE)
125 Dedicated Registers 130 Direct Memory Access (DMA)
140 matrix transfer device 141 address generator 142 bank number generator 150 first local memory 153 first data selection unit 160 second local memory 161 RAM bank 162 second data selection unit 163 data ring 170 external memory 180 instruction memory

Claims (7)

a central processing unit;
a vector processing unit electronically coupled to the central processing unit and configured to perform operations based on instructions received from the central processing unit;
an instruction memory unit electronically connected to the central processing unit and configured to store instructions;
an external memory unit;
a first local memory unit electronically connected to the central processing unit and configured to store one-dimensional systolic data;
a second local memory unit electronically connected to the vector processing unit and configured to store matrix data;
a direct memory electronically coupled to the first local memory unit, the second local memory unit, the instruction memory unit, and the external memory unit and configured to access data in the external memory unit; an access unit;
a matrix transfer unit electronically connected between the first and second local memory units and configured to transfer data between the first and second local memory units;
with
data is transferred through the direct memory access unit with timing based on a predetermined selection priority;
data processing system. The matrix transfer unit is capable of performing matrix transposition on the data when transferring data between the first and second local memory units.
The data processing system of claim 1 . The matrix transfer unit comprises an address generator configured to generate memory addresses for a source memory and a destination memory, the source memory being one of the first local memory unit and the second local memory unit. one and the destination memory is the other of the first local memory unit and the second local memory unit;
3. The data processing system of claim 2 . The second local memory unit has a data ring configured to ring broadcast data, the data ring using the predetermined selection priority when multiple memory requests are received. transferring data to the first local memory unit and the external memory unit via the direct memory access unit using
4. A data processing system according to any one of claims 1-3 . The first and second local memory units each include a plurality of 2-port RAM banks in which data is stored, and the number of 2-port RAM banks in the first local memory unit is equal to the number of the 2-port RAM banks in the second local memory. equal to the number of said two-port RAM banks of a unit;
5. A data processing system according to any one of claims 1-4 . A method for a data processing system, said data processing system comprising: a central processing unit; a vector processing unit electronically connected to said central processing unit; a memory unit; an external memory unit; a first local memory unit electronically connected to said central processing unit; a second local memory unit electronically connected to said vector processing unit; a direct memory access unit electronically connected to the local memory unit, the second local memory unit, the instruction memory unit, and the external memory unit; and electronically between the first and second local memory units a matrix transfer unit connected to and configured to transfer data between the first and second local memory units, the method comprising:
performing operations by the vector processing unit based on instructions received from the central processing unit;
storing instructions by the instruction memory unit;
storing one-dimensional systolic data by the first local memory unit;
storing matrix data by the second local memory unit;
accessing data in the external memory unit by the direct memory access unit;
the matrix transfer unit performing matrix transposition on the data when transferring data between the first and second local memory units;
including
data is transferred through the direct memory access unit with timing based on a predetermined selection priority;
Method. A program for a data processing system, said data processing system comprising: a central processing unit; a vector processing unit electronically connected to said central processing unit; a memory unit; an external memory unit; a first local memory unit electronically connected to said central processing unit; a second local memory unit electronically connected to said vector processing unit; a direct memory access unit electronically connected to the local memory unit, the second local memory unit, the instruction memory unit, and the external memory unit; and electronically between the first and second local memory units a matrix transfer unit connected to and configured to transfer data between the first and second local memory units, the program comprising:
causing the vector processing unit to perform operations based on instructions received from the central processing unit;
storing instructions in the instruction memory unit;
causing the first local memory unit to store one-dimensional systolic data;
storing matrix data in the second local memory unit;
causing the direct memory access unit to access data in the external memory unit;
causing the matrix transfer unit to perform a matrix transpose on the data when transferring the data between the first and second local memory units;
data is transferred through the direct memory access unit with timing based on a predetermined selection priority;
program.

Images