WO2024169293A1 - 计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统 - Google Patents

计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统 Download PDF

Info

Publication number
WO2024169293A1
WO2024169293A1 PCT/CN2023/132743 CN2023132743W WO2024169293A1 WO 2024169293 A1 WO2024169293 A1 WO 2024169293A1 CN 2023132743 W CN2023132743 W CN 2023132743W WO 2024169293 A1 WO2024169293 A1 WO 2024169293A1
Authority
WO
WIPO (PCT)
Prior art keywords
computing
matrix
matrix multiplication
memory
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/132743
Other languages
English (en)
French (fr)
Inventor
李仁刚
杨宏斌
董刚
蒋东东
曹其春
胡克坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to US19/154,296 priority Critical patent/US12608319B1/en
Publication of WO2024169293A1 publication Critical patent/WO2024169293A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0877—Cache access modes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the field of computer technology, and in particular to a computing core, an accelerator, a computing method, a device, an equipment, a non-volatile readable storage medium and a system.
  • the purpose of this application is to provide a computing core, an accelerator, a computing method, an apparatus, a device, a non-volatile readable storage medium and a system to improve the computing speed and efficiency of matrix multiplication operations in graph neural networks.
  • the scheme is as follows:
  • the present application provides a computing core, comprising: a first cache device, a second cache device, a matrix multiplication computing device, and an output device;
  • a first cache device used to load data in a first matrix participating in a current matrix multiplication operation in the graph neural network from an external first memory according to a received computing instruction
  • a second cache device used for loading data in a second matrix participating in a current matrix multiplication operation from an external second memory according to a calculation instruction
  • a matrix multiplication calculation device used for calculating in parallel the matrix multiplication result of N row data or N column data in the first matrix and N column data or N row data in the second matrix according to the parallel number N set by the calculation instruction;
  • An output device used for temporarily storing the matrix multiplication result, and when the data amount of the matrix multiplication result meets the preset writing condition, writing the matrix multiplication result into the first memory or the second memory set by the calculation instruction;
  • the transmission bandwidth of the second memory is greater than that of the first memory; if the matrix multiplication result needs to be calculated in a traversal manner when participating in each round of calculation in the next matrix multiplication operation, the calculation instruction is set with the second memory; otherwise, the calculation instruction is set with the first memory.
  • the first cache device is configured to: load remaining unloaded data in the first matrix participating in the current matrix multiplication operation from the first memory during the process of the matrix multiplication calculation device calculating the matrix multiplication result;
  • the second cache device is configured to load the remaining unloaded data in the second matrix participating in the current matrix multiplication operation from the second memory during the process of the matrix multiplication calculation device calculating the matrix multiplication result.
  • the first cache device loads the data in the first matrix using a ping-pong method.
  • the second cache device uses a first-in-first-out queue to load data in the second matrix.
  • the matrix multiplication computing device includes: at least one computing unit, the computing unit is composed of a multiplier and an accumulator and the amount of data calculated each time is equal to N.
  • the present application provides an accelerator, comprising: a controller, a first memory, a second memory having a transmission bandwidth greater than that of the first memory, and the computing core described above;
  • the controller is connected to the first memory, the second memory, and the computing core respectively;
  • the computing core is connected to the first memory and the second memory respectively;
  • a controller configured to send a computation instruction for a matrix multiplication operation in a graph neural network to a first memory, a second memory, and a computation core;
  • a first memory used for acquiring and storing a first matrix involved in a current matrix multiplication operation in a direct memory access (DMA) manner according to a calculation instruction;
  • DMA direct memory access
  • a second memory is used to obtain and store a second matrix involved in the current matrix multiplication operation in a DMA manner according to the calculation instruction;
  • the computing core is used to calculate the matrix multiplication result of the first matrix and the second matrix according to the computing instruction.
  • controller is also used to: control multiple computing cores to perform the same matrix multiplication operation.
  • the controller is configured to: when controlling multiple computing cores to perform the same matrix multiplication operation, enable each computing core to calculate the matrix multiplication result of the same row data or the same column data in the first matrix and different column data or different row data in the second matrix within the same computing cycle.
  • the controller is configured to: when controlling multiple computing cores to perform the same matrix multiplication operation, enable each computing core to respectively calculate the matrix multiplication results of different column data or different row data in the first matrix and the same column data or the same row data in the second matrix within the same computing cycle.
  • controller is further used to: control different computing cores to perform different matrix multiplication operations in the same computing cycle.
  • the computation instructions are used to implement arbitrary matrix multiplication operations in graph neural networks.
  • the computing core includes: a first cache device, a second cache device, a matrix multiplication computing device and an output device;
  • a first cache device used to load data in a first matrix participating in a current matrix multiplication operation in the graph neural network from an external first memory according to a received computing instruction
  • a second cache device used for loading data in a second matrix participating in a current matrix multiplication operation from an external second memory according to a calculation instruction
  • a matrix multiplication calculation device used for calculating in parallel the matrix multiplication result of N row data or N column data in the first matrix and N column data or N row data in the second matrix according to the parallel number N set by the calculation instruction;
  • An output device used for temporarily storing the matrix multiplication result, and when the data amount of the matrix multiplication result meets the preset writing condition, writing the matrix multiplication result into the first memory or the second memory set by the calculation instruction;
  • the transmission bandwidth of the second memory is greater than that of the first memory; if the matrix multiplication result needs to be calculated in a traversal manner when participating in each round of calculation in the next matrix multiplication operation, the calculation instruction is set with the second memory; otherwise, the calculation instruction is set with the first memory.
  • the first cache device is configured to: load remaining unloaded data in the first matrix participating in the current matrix multiplication operation from the first memory during the process of the matrix multiplication calculation device calculating the matrix multiplication result;
  • the second cache device is configured to load the remaining unloaded data in the second matrix participating in the current matrix multiplication operation from the second memory during the process of the matrix multiplication calculation device calculating the matrix multiplication result.
  • the first cache device loads the data in the first matrix using a ping-pong method.
  • the second cache device uses a first-in-first-out queue to load data in the second matrix.
  • the matrix multiplication computing device includes: at least one computing unit, the computing unit is composed of a multiplier and an accumulator and the amount of data calculated each time is equal to N.
  • the present application provides a calculation method, comprising:
  • the calculation instruction is provided with a parallel number N and a write position of the matrix multiplication result of the corresponding matrix multiplication operation;
  • the computing instructions are sent to the accelerator mentioned above so that the accelerator can accelerate the computing of the graph neural network.
  • sending the computing instruction to the accelerator mentioned above includes:
  • the corresponding calculation instructions are sent to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • it also includes:
  • the value of the parallel number N is determined according to the resource configuration of the computing core in the accelerator.
  • the value of the parallel number N is determined according to the resource configuration of the computing core in the accelerator, including:
  • the constraint formula is constructed: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; where Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f represents the computing clock frequency of the computing core;
  • N is determined based on the constraint formula.
  • the present application provides a computing device, including:
  • a generation module used to generate a calculation instruction for each matrix multiplication operation in the graph neural network; the calculation instruction is provided with a parallel number N and a writing position of the matrix multiplication result of the corresponding matrix multiplication operation;
  • the sending module is used to send the computing instructions to the accelerator mentioned above so that the accelerator can accelerate the computing of the graph neural network.
  • the sending module is configured to:
  • the corresponding calculation instructions are sent to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • it also includes:
  • the configuration module is used to determine the value of the parallel number N according to the resource configuration of the computing core in the accelerator.
  • the configuration module is set to:
  • the constraint formula is constructed: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; where Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f represents the computing clock frequency of the computing core;
  • N is determined based on the constraint formula.
  • the present application provides an electronic device, including:
  • the processor is used to execute the computer program to implement the aforementioned disclosed computing method.
  • the present application provides a non-volatile readable storage medium for storing a computer program, wherein the computer program implements the aforementioned disclosed computing method when executed by a processor.
  • the present application provides a computing system, comprising: a server and at least one accelerator disclosed above.
  • the present application provides a computing core, including: a first cache device, a second cache device, a matrix multiplication computing device and an output device;
  • the first cache device is used to load the data in the first matrix participating in the current matrix multiplication operation in the graph neural network from an external first memory according to the received computing instruction;
  • the second cache device is used to load the data in the second matrix participating in the current matrix multiplication operation from an external second memory according to the computing instruction;
  • the matrix multiplication computing device is used to parallelly calculate the matrix multiplication result of N row data or N column data in the first matrix and N column data or N row data in the second matrix according to the parallel number N set by the computing instruction;
  • the output device is used to temporarily store the matrix multiplication result, and when the data amount of the matrix multiplication result meets the preset write condition, the matrix multiplication result is written to the first memory or the second memory set by the computing instruction; wherein the transmission bandwidth of the second memory is greater than that of the first memory; if the matrix multiplication result needs to participate in the calculation in
  • the computing core provided by the present application can calculate the matrix multiplication results of N row data or N column data in the first matrix and N column data or N row data in the second matrix in parallel, so N final matrix multiplication results can be obtained at one time, which improves the computing efficiency and speed; the computing core does not need to temporarily store intermediate results, saving on-chip resources.
  • the present application can determine which memory to store the matrix multiplication result according to the participation mode of the matrix multiplication result in each round of calculation in the next matrix multiplication operation.
  • the matrix multiplication result participates in each round of calculation in the next matrix multiplication operation in a traversal manner, the matrix multiplication result is stored in the second memory; otherwise, the matrix multiplication result is stored in the first memory.
  • the matrix multiplication result is written, which can reduce the number of writes and improve write efficiency and bandwidth utilization.
  • an accelerator, computing method, device, equipment, non-volatile readable storage medium and system provided by the present application also have the above-mentioned technical effects.
  • FIG1 is a schematic diagram of a computing core disclosed in the present application.
  • FIG2 is a schematic diagram of a computing unit structure disclosed in the present application.
  • FIG3 is a schematic diagram of an accelerator structure disclosed in the present application.
  • FIG4 is a schematic diagram of another accelerator structure disclosed in the present application.
  • FIG5 is a schematic diagram of the structure of a third accelerator disclosed in this application.
  • FIG6 is a flow chart of a calculation method disclosed in the present application.
  • FIG7 is a schematic diagram of a data parallel input disclosed in the present application.
  • FIG8 is a schematic diagram of a 4*4*4 PE array disclosed in the present application.
  • FIG9 is a schematic diagram of a continuous storage of written data disclosed in the present application.
  • FIG10 is a schematic diagram of a matrix multiplication calculation process disclosed in the present application.
  • FIG. 11 is a schematic diagram of an electronic device disclosed in the present application.
  • the present application provides a solution that can improve the computing speed and efficiency of matrix multiplication operations in graph neural networks.
  • an embodiment of the present application discloses a computing core, including: a first cache device, a second cache device, a matrix multiplication computing device, and an output device.
  • the first cache device is configured to load data in a first matrix participating in a current matrix multiplication operation in the graph neural network from an external first memory according to a received computing instruction; the second cache device is configured to load data in a first matrix participating in a current matrix multiplication operation in the graph neural network from an external second memory according to a computing instruction.
  • the matrix multiplication calculation device is configured to calculate the matrix multiplication result of N row data or N column data in the first matrix and N column data or N row data in the second matrix in parallel according to the parallel number N set by the calculation instruction;
  • the output device is configured to temporarily store the matrix multiplication result, and when the data amount of the matrix multiplication result meets the preset writing condition, the matrix multiplication result is written into the first memory or the second memory set by the calculation instruction.
  • one row of data is: a row of data in the matrix;
  • one column of data is: a column of data in the matrix.
  • the transmission bandwidth of the second memory is greater than that of the first memory (such as DDR); if the matrix multiplication result needs to participate in the calculation in a traversal manner in each round of calculation in the next matrix multiplication operation, the calculation instruction is set with the second memory; otherwise, the calculation instruction is set with the first memory.
  • HBM High Bandwidth Memory
  • DDR Double Data Rate
  • the calculation instruction is used to implement any matrix multiplication operation in the graph neural network, and the value of the parallel number N is determined according to the resource configuration of the computing core.
  • a constraint formula is constructed based on the resource configuration of the computing core: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; where Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f is the computing clock frequency of the computing core; the value of N is determined based on the constraint formula.
  • the first cache device is configured to: load the remaining unloaded data in the first matrix participating in the current matrix multiplication operation from the first memory during the process of the matrix multiplication computing device calculating the matrix multiplication result; correspondingly, the second cache device is configured to: load the remaining unloaded data in the second matrix participating in the current matrix multiplication operation from the second memory during the process of the matrix multiplication computing device calculating the matrix multiplication result.
  • the first cache device loads the data in the first matrix using a ping-pong method.
  • the first cache device may be a URAM (Ultra RAM).
  • the second cache device uses a first-in-first-out queue to load data in the second matrix.
  • the second cache device uses a first first-in-first-out queue to cache data in the second matrix and uses a second first-in-first-out queue to cache matrix multiplication results.
  • the matrix multiplication computing device includes: at least one computing unit, the computing unit is composed of a multiplier and an accumulator, and the amount of data calculated each time is equal to N.
  • Any computing unit may include at least one multiplier group, and a multiplier group includes two multipliers; and the output of a multiplier group is connected to the accumulator. As shown in Figure 2, the amount of data calculated each time is 4, so there are 4 multiplier groups. If necessary, the multiplier group and the accumulator connected to its output can be flexibly added or deleted, thereby realizing flexible expansion and adjustment of the computing device.
  • the quantization interception in Figure 2 is used to realize: quantization and interception of data bit width. If the data bit width is 8, then the bit width of the multiplication result is 16. After accumulation, the data bit width will be larger. At this time, the data bit width of the final result needs to be intercepted to 8 bits.
  • the computing core provided in this embodiment can calculate the matrix multiplication results of N row data or N column data in the first matrix and N column data or N row data in the second matrix in parallel, so N final matrix multiplication results can be obtained at one time, improving the computing efficiency and speed; the computing core also does not need to temporarily store intermediate results, saving on-chip resources.
  • the present application can determine which memory to store the matrix multiplication result according to the way the matrix multiplication result participates in each round of calculation in the next matrix multiplication operation.
  • the matrix multiplication result is stored in the second memory; otherwise, the matrix multiplication result is stored in the first memory.
  • Implementation Make the storage format of the matrix multiplication result in the memory consistent with the output format when it participates in the calculation, so as to facilitate the sequential reading of data in the continuous calculation process, and there is no need to perform matrix transposition, so as to reduce the time overhead of memory access and improve efficiency.
  • the preset writing conditions are met, the matrix multiplication result is written, which can reduce the number of writes and improve the writing efficiency and bandwidth utilization. It can be seen that this embodiment can improve the calculation speed and efficiency of the matrix multiplication operation in the graph neural network.
  • An accelerator provided in an embodiment of the present application is introduced below.
  • the accelerator described below and the computing core described above can refer to each other.
  • An embodiment of the present application discloses an accelerator, including: a controller, a first memory, a second memory having a transmission bandwidth greater than that of the first memory, and the computing core described above.
  • the controller is connected to the first memory, the second memory, and the computing core respectively; the computing core is connected to the first memory and the second memory respectively.
  • the controller is configured to send the calculation instructions of the matrix multiplication operation in the graph neural network to the first memory, the second memory, and the computing core;
  • the first memory is configured to obtain and store the first matrix involved in the current matrix multiplication operation in a direct memory access (DMA) (Direct Memory Access) manner according to the calculation instructions;
  • the second memory is configured to obtain and store the second matrix involved in the current matrix multiplication operation in a DMA manner according to the calculation instructions;
  • the computing core is configured to calculate the matrix multiplication result of the first matrix and the second matrix according to the calculation instructions.
  • DMA direct memory access
  • the transmission bandwidth of the second memory (such as HBM) is greater than that of the first memory (such as DDR), and the matrix participating in the calculation in a traversal manner is stored in the second memory, which can make the reading speed of the matrix participating in the calculation in a traversal manner faster and improve the calculation speed.
  • the computing core includes: a first cache device, a second cache device, a matrix multiplication computing device and an output device; the first cache device can load the remaining unloaded data in the first matrix participating in the current matrix multiplication operation from the first memory during the process of the matrix multiplication computing device calculating the matrix multiplication result; correspondingly, the second cache device can load the remaining unloaded data in the second matrix participating in the current matrix multiplication operation from the second memory during the process of the matrix multiplication computing device calculating the matrix multiplication result.
  • data can be loaded in advance without waiting for the previous operation to be completed, and the computing device will not be idle, and data calculation can be continuously performed, thereby improving the computing resource utilization and system computing efficiency and reducing task delays.
  • a plurality of computing cores may be provided in the accelerator; accordingly, a scheduling core needs to be provided to enable the controller to schedule and control these computing cores.
  • the controller is further configured to: control multiple computing cores to perform the same matrix multiplication operation.
  • the controller is configured to: when controlling multiple computing cores to perform the same matrix multiplication operation, each computing core calculates the matrix multiplication result of the same row data or the same column data in the first matrix and the different column data or different row data in the second matrix in the same calculation cycle.
  • computing core 1 calculates the multiplication result of the first row in the first matrix and the first column in the second matrix
  • computing core 2 calculates the multiplication result of the first row in the first matrix and the second column in the second matrix, thereby achieving: the reuse of the first row data in the first matrix on multiple computing cores can avoid repeated reading of data and improve computing efficiency.
  • row data 1 (row 1 data) in the S matrix is reused by computing core 1 and computing core 2.
  • the controller is configured to: when controlling multiple computing cores to perform the same matrix multiplication operation, each computing core calculates the matrix multiplication results of different column data or different row data in the first matrix and the same column data or the same row data in the second matrix in the same computing cycle. For example: in the same computing cycle (the same time period), computing core 1 calculates the multiplication result of the first row in the first matrix and the first column in the second matrix, and computing core 2 calculates the multiplication result of the second row in the first matrix and the first column in the second matrix, thereby achieving: the reuse of the first row of data in the second matrix on multiple computing cores can avoid repeated reading of data and improve computing efficiency. As shown in Figure 4, the traversal matrix is reused by computing core 1 and computing core 2.
  • the controller is further configured to: control different computing cores to perform different matrix multiplication operations in the same computing cycle
  • calculation core 1 calculates the multiplication result of the first row in the first matrix and the first column in the second matrix
  • calculation core 2 calculates the multiplication result of the first row in the third matrix and the first column in the fourth matrix.
  • This can achieve: the matrix multiplication operation of the first matrix and the second matrix, and the matrix multiplication operation of the third matrix and the fourth matrix are performed simultaneously, which can improve the calculation efficiency.
  • calculation core 1 calculates the multiplication result of matrix A and matrix B
  • calculation core 2 calculates the multiplication result of matrix C and matrix D.
  • a high-bandwidth HBM can be used to implement data caching of the traversal matrix; and a small number of rows/columns of data in the S matrix can be cached using a single-port low-bandwidth DDR4.
  • the use of high- and low-bandwidth memories can hide the memory access time overhead and improve the overall throughput efficiency of the system because subsequent data can be continuously loaded during calculation.
  • the calculation instruction is used to implement any matrix multiplication operation in the graph neural network.
  • the main operation in the graph neural network is: the feature matrix F is multiplied by the corresponding wavelet matrix Way and the weight matrix W, which is expressed by the formula: Way ⁇ (F ⁇ W).
  • Way ⁇ (F ⁇ W) As shown in the formula, when the feature matrix F is multiplied by W, F is the left matrix on the left side of the multiplication sign, and when the product of F ⁇ W is multiplied by Way, the product is the right matrix on the right side of the multiplication sign.
  • This position transformation requires that the product of F ⁇ W is determined in advance according to the position of the product in the next multiplication operation. Whether the product is stored in row priority or column priority.
  • the previous matrix multiplication process is: keep the N rows of data in the left matrix unchanged, traverse all columns of the right matrix, and obtain the result of N rows multiplied by all columns; calculate the result of N+1 ⁇ 2N rows multiplied by all columns again until the calculation is completed; this process reuses the row data of the left matrix while generating the row priority matrix in sequence.
  • the previous matrix multiplication process is: keep the N columns of data in the right matrix unchanged, traverse all rows of the left matrix, and obtain the result of multiplying all rows by N columns; then calculate the result of multiplying all rows by N+1 ⁇ 2N columns again until the calculation is completed; this process reuses the column data of the right matrix while generating the column-major matrix in sequence.
  • any computing core includes: a first cache device, a second cache device, a matrix multiplication computing device and an output device;
  • the matrix multiplication computing device includes: at least one computing unit, which is composed of a multiplier and an accumulator and the amount of data calculated each time is equal to N.
  • the calculation acceleration ratio can be adjusted by flexibly setting the value of N.
  • the value of the parallel number N is determined according to the resource configuration of the computing core. For example: construct a constraint formula based on the resource configuration of the computing core: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; wherein Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f is the computing clock frequency of the computing core; the value of N is determined based on the constraint formula.
  • this embodiment provides an accelerator, which can avoid the memory access and time overhead of matrix transposition, realize data reuse in the calculation process, improve the calculation speed and efficiency of matrix multiplication operations in graph neural networks, and realize flexible parallel expansion of computing devices.
  • the present application embodiment discloses a calculation method, including:
  • sending the calculation instructions to the accelerator in the above text includes: sending the corresponding calculation instructions to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • the value of the parallel number N is determined according to the resource configuration of the computing core in the accelerator. In an optional implementation, the value of the parallel number N is determined according to the resource configuration of the computing core in the accelerator, including: constructing a constraint formula based on the resource configuration of the computing core: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; wherein Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f is the computing clock frequency of the computing core; the value of N is determined based on the constraint formula.
  • the maximum N value that satisfies Count ⁇ N3/2 and BW ⁇ DW ⁇ f ⁇ N2 is solved as the parallel number.
  • this embodiment provides a calculation method that can avoid the memory access and time overhead of matrix transposition, realize data reuse in the calculation process, improve the calculation speed and efficiency of matrix multiplication operations in graph neural networks, and realize flexible parallel expansion of computing devices.
  • a computing device provided in an embodiment of the present application is introduced below.
  • the computing device described below and other embodiments may be referenced to each other.
  • the present application embodiment discloses a computing device, including:
  • a generation module is configured to generate a calculation instruction for each matrix multiplication operation in the graph neural network; the calculation instruction is provided with a parallel number N and a write position of a matrix multiplication result of the corresponding matrix multiplication operation;
  • the sending module is configured to send the computing instructions to the accelerator mentioned above so that the accelerator can accelerate the computing of the graph neural network.
  • the sending module is configured to:
  • the corresponding calculation instructions are sent to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • it also includes:
  • the configuration module is configured to determine the value of the parallel number N according to the resource configuration of the computing core in the accelerator.
  • the configuration module is configured to:
  • the constraint formula is constructed: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; where Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f represents the computing clock frequency of the computing core;
  • N is determined based on the constraint formula.
  • this embodiment provides a computing device that can avoid the memory access and time overhead of matrix transposition, realize data reuse in the calculation process, improve the calculation speed and efficiency of matrix multiplication operations in graph neural networks, and realize flexible parallel expansion of computing devices.
  • a design scheme for a computing device for matrix multiplication operations in a graph neural network is provided below.
  • a row of data or a column of data in the matrix is divided into 7 data blocks of size K, and the PE array shown in FIG7 can simultaneously calculate the data input thereto.
  • 7 K-sized data blocks illustrated in FIG7 are only examples. In actual implementation, the number of data blocks should be an exponent of 2, that is, a row of data or a column of data is divided into 2 exponential K-sized data blocks.
  • kernel is the smallest unit involved in calculation, and a single PE adopts a multiplier and adder structure, which has the advantage of being able to accelerate the calculation process in parallel. If the length of a row is L, then the time for sequential calculation of the entire row/column is L, while using the structure shown in FIG2, the time taken is log2(L).
  • a 4*4*4 PE array is shown in Figure 8.
  • Figure 8 there is a group of data multiplexing in each PE.
  • the computing core provided in this embodiment can calculate the matrix multiplication results of N row data or N column data in the first matrix and N column data or N row data in the second matrix in parallel, so N final matrix multiplication results can be obtained at one time, improving the computing efficiency and speed; the computing core also does not need to temporarily store intermediate results, saving on-chip resources.
  • the matrix multiplication operation process implemented according to this embodiment includes:
  • the result is written to the corresponding external HBM/DDR4 through bus burst transmission, which can reduce the number of writes.
  • write to the external memory once when a burst length is sufficient.
  • the burst length of AXI4 Advanced eXtensible Interface 4, a bus protocol
  • the int8 data type 4096 data need to be gathered to write to the external memory once.
  • this embodiment uses a quantitative design method to obtain the organization form and scale of the PE array based on on-chip resources and externally available memory bandwidth; uses a row/column priority matrix generation method to generate a feature matrix in the format required for the next step of the graph neural network calculation; uses a complete hardware
  • the acceleration system and method are used to load data, perform traversal calculations, and write back results; the flexibly expandable multi-PE array system architecture is used to accelerate the parallel calculation of single or multiple matrix multiplications; the operation instructions are split into data loading, calculation, and result write back instructions, and data is loaded in advance to ensure that the PE array works continuously and uninterruptedly.
  • This solution can flexibly adapt hardware resources, realize periodic pipeline processing of instructions, effectively improve calculation and memory access efficiency, and complete flexible transformation and continuous and efficient calculation of matrices at all levels in graph neural networks.
  • an electronic device including:
  • a memory 1101 is configured to store a computer program
  • the processor 1102 is configured to execute a computer program to implement the method disclosed in any of the above embodiments.
  • a calculation instruction is generated for each matrix multiplication operation in the graph neural network; the calculation instruction is provided with a parallel number N and a write location of the matrix multiplication result of the corresponding matrix multiplication operation; the calculation instruction is sent to the accelerator mentioned above so that the accelerator can accelerate the calculation of the graph neural network.
  • the processor executes a computer program stored in the memory
  • the following steps can be implemented: the corresponding calculation instructions are sent to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • the processor executes the computer program stored in the memory
  • the following steps may be implemented: determining the value of the parallel number N according to the resource configuration of the computing core in the accelerator.
  • the processor executes a computer program stored in the memory
  • the following steps can be implemented: construct a constraint formula based on the resource configuration of the computing core: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; wherein Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f is the computing clock frequency of the computing core; determine the value of N based on the constraint formula.
  • an embodiment of the present application further provides a server as the above-mentioned electronic device.
  • the server may include: at least one processor, at least one memory, a power supply, a communication interface, an input/output interface, and a communication bus.
  • the memory is configured to store a computer program, which is loaded and executed by the processor to implement the relevant steps in the computing method disclosed in any of the above-mentioned embodiments.
  • the power supply is configured to provide operating voltage for each hardware device on the server;
  • the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application and is not limited here;
  • the input and output interface is configured to obtain external input data or output data to the outside world, and its interface type can be selected according to application needs and is not limited here.
  • the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc.
  • the resources stored thereon include operating system, computer programs and data, etc.
  • the storage method can be temporary storage or permanent storage.
  • the operating system is configured to manage and control the hardware devices and computer programs on the server to realize the operation and processing of the data in the memory by the processor, and it can be Windows Server, Netware, Unix, Linux, etc.
  • computer programs can also include computer programs that can be configured to complete other specific tasks.
  • data can also include data such as developer information of virtual machines.
  • the embodiment of the present application further provides a terminal as the above electronic device.
  • the terminal may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.
  • the terminal in this embodiment includes: a processor and a memory.
  • the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc.
  • the processor may be a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array,
  • the processor may also include a main processor and a coprocessor.
  • the main processor is a processor configured to process data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor configured to process data in a standby state.
  • the processor may be integrated with a GPU (Graphics Processing Unit), which is configured to be responsible for rendering and drawing the content to be displayed on the display screen.
  • the processor may also include an AI (Artificial Intelligence) processor, which is configured to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the memory may include one or more non-volatile readable storage media, which may be non-transitory.
  • the memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices.
  • the memory is at least configured to store the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps in the computing method performed by the terminal side disclosed in any of the aforementioned embodiments.
  • the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage.
  • the operating system may include Windows, Unix, Linux, etc.
  • the data may include, but is not limited to, update information of the application.
  • the terminal may also include a display screen, an input and output interface, a communication interface, a sensor, a power supply, and a communication bus.
  • a non-volatile readable storage medium provided in an embodiment of the present application is introduced below.
  • the non-volatile readable storage medium described below can be referenced with other embodiments.
  • the embodiment of the present application discloses a non-volatile readable storage medium, which is configured to store a computer program, wherein the computer program, when executed by a processor, implements the computing method disclosed in the above embodiment.
  • the non-volatile readable storage medium as a carrier for storing resources, may be a read-only memory, a random access memory, a disk or an optical disk, etc.
  • the resources stored thereon include an operating system, a computer program and data, etc., and the storage method may be temporary storage or permanent storage.
  • the computer program executed by the processor can implement the following steps: generate a calculation instruction for each matrix multiplication operation in the graph neural network; the calculation instruction is provided with a parallel number N and a write location of the matrix multiplication result of the corresponding matrix multiplication operation; the calculation instruction is sent to the accelerator mentioned above, so that the accelerator accelerates the calculation of the graph neural network.
  • the computer program executed by the processor can implement the following steps: send the corresponding calculation instructions to the accelerator in sequence according to the calculation order of different matrix multiplication operations in the graph neural network.
  • the computer program executed by the processor can implement the following steps: determining the value of the parallel number N according to the resource configuration of the computing core in the accelerator.
  • the computer program executed by the processor can implement the following steps: construct a constraint formula based on the resource configuration of the computing core: Count ⁇ N3/2, BW ⁇ DW ⁇ f ⁇ N2; wherein Count represents the number of computing resources of the computing core, BW represents the memory access bandwidth of the computing core, DW represents the data bit width of the computing core, and f is the computing clock frequency of the computing core; determine the value of N based on the constraint formula.
  • a computing system provided in an embodiment of the present application is introduced below.
  • the computing system described below and other embodiments may be cross-referenced.
  • the embodiment of the present application discloses a computing system, including: a server and at least one of the aforementioned disclosed accelerators.
  • the server is used to generate a computing instruction for each matrix multiplication operation in a graph neural network; the computing instruction is provided with a parallel number N and a write location of the matrix multiplication result of the corresponding matrix multiplication operation; and the computing instruction is sent to any accelerator.
  • the accelerator that receives the computing instruction performs accelerated computing on the graph neural network.
  • this embodiment provides a computing system that can avoid the memory access and time overhead of matrix transposition, realize data reuse in the calculation process, improve the calculation speed and efficiency of matrix multiplication operations in graph neural networks, and realize flexible parallel expansion of computing devices.
  • the steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two.
  • the software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile readable storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Neurology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Complex Calculations (AREA)

Abstract

本申请公开了计算机技术领域内的一种计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统。本申请能够并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果，因此可一次性得到N个最终矩阵乘结果，提高了计算效率和速度；计算核也无需暂存中间结果，节约了片上资源。在得到矩阵乘结果后，本申请能够按照该矩阵乘结果参与下一矩阵乘操作中的每一轮计算时的参与方式确定将该矩阵乘结果存入哪个存储器，由此可实现：使矩阵乘结果在存储器中的存储格式与其参与计算时的输出格式一致，便于在连续的计算过程中顺序读取数据，而无需进行矩阵转置，因此还能降低访存的时间开销，提高效率。

Description

计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统
相关申请的交叉引用
本申请要求于2023年2月15日提交中国专利局，申请号为2023101146404，申请名称为“计算核、加速器、计算方法、装置、设备、介质及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统。
背景技术
目前，对于图神经网络中的矩阵乘，现有的计算器件可以采用矩阵分块操作进行矩阵乘操作，但此方式需要将大量的中间结果暂存到计算器件之外，计算器件在进行后续计算时再读回至片内进行累加，增加了许多额外的数据搬移操作，会浪费计算资源，降低计算速度。
因此，如何提高图神经网络中矩阵乘操作的计算速度和计算效率，是本领域技术人员需要解决的问题。
发明内容
有鉴于此，本申请的目的在于提供一种计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统，以提高图神经网络中矩阵乘操作的计算速度和计算效率。其方案如下：
第一方面，本申请提供了一种计算核，包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；
第一缓存器件，用于根据接收到的计算指令从外部的第一存储器中加载参与图神经网络中的当前矩阵乘操作的第一矩阵中的数据；
第二缓存器件，用于根据计算指令从外部的第二存储器中加载参与当前矩阵乘操作的第二矩阵中的数据；
矩阵乘计算器件，用于按照计算指令设定的并行数N并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果；
输出器件，用于暂存矩阵乘结果，在矩阵乘结果的数据量满足预设写条件时，将矩阵乘结果写入计算指令设定的第一存储器或第二存储器；
其中，第二存储器的传输带宽大于第一存储器；若矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则计算指令设定有第二存储器；否则，计算指令设定有第一存储器。
可选地，第一缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第一存储器中加载参与当前矩阵乘操作的第一矩阵中的剩余未加载数据；
相应地，第二缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第二存储器中加载参与当前矩阵乘操作的第二矩阵中的剩余未加载数据。
可选地，第一缓存器件利用乒乓方式加载第一矩阵中的数据。
可选地，第二缓存器件利用先入先出队列加载第二矩阵中的数据。
可选地，矩阵乘计算器件包括：至少一个计算单元，计算单元利用乘法器和累加器构成且每次计算的数据量等于N。
第二方面，本申请提供了一种加速器，包括：控制器、第一存储器、传输带宽大于第一存储器的第二存储器、以及前文的计算核；
其中，控制器与第一存储器、第二存储器、计算核分别连接；计算核与第一存储器、第二存储器分别连接；
控制器，用于将图神经网络中的矩阵乘操作的计算指令发送至第一存储器、第二存储器和计算核；
第一存储器，用于根据计算指令以直接内存访问DMA方式获取并存储参与当前矩阵乘操作的第一矩阵；
第二存储器，用于根据计算指令以DMA方式获取并存储参与当前矩阵乘操作的第二矩阵；
计算核，用于根据计算指令计算第一矩阵和第二矩阵的矩阵乘结果。
可选地，计算核有多个。
可选地，控制器还用于：控制多个计算核执行同一矩阵乘操作。
可选地，控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内计算第一矩阵中的相同行数据或相同列数据与第二矩阵中的不同列数据或不同行数据的矩阵乘结果。
可选地，控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内分别计算第一矩阵中的不同列数据或不同行数据与第二矩阵中的相同列数据或相同行数据的矩阵乘结果。
可选地，控制器还用于：控制不同计算核在同一计算周期内执行不同矩阵乘操作。
可选地，计算指令用于实现图神经网络中的任意矩阵乘操作。
可选地，计算核包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；
第一缓存器件，用于根据接收到的计算指令从外部的第一存储器中加载参与图神经网络中的当前矩阵乘操作的第一矩阵中的数据；
第二缓存器件，用于根据计算指令从外部的第二存储器中加载参与当前矩阵乘操作的第二矩阵中的数据；
矩阵乘计算器件，用于按照计算指令设定的并行数N并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果；
输出器件，用于暂存矩阵乘结果，在矩阵乘结果的数据量满足预设写条件时，将矩阵乘结果写入计算指令设定的第一存储器或第二存储器；
其中，第二存储器的传输带宽大于第一存储器；若矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则计算指令设定有第二存储器；否则，计算指令设定有第一存储器。
可选地，第一缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第一存储器中加载参与当前矩阵乘操作的第一矩阵中的剩余未加载数据；
相应地，第二缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第二存储器中加载参与当前矩阵乘操作的第二矩阵中的剩余未加载数据。
可选地，第一缓存器件利用乒乓方式加载第一矩阵中的数据。
可选地，第二缓存器件利用先入先出队列加载第二矩阵中的数据。
可选地，矩阵乘计算器件包括：至少一个计算单元，计算单元利用乘法器和累加器构成且每次计算的数据量等于N。
第三方面，本申请提供了一种计算方法，包括：
针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；
将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
可选地，将计算指令发送至前文的加速器，包括：
按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
可选地，还包括：
根据加速器中的计算核的资源配置确定并行数N的取值。
可选地，根据加速器中的计算核的资源配置确定并行数N的取值，包括：
基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；
基于约束公式确定N的取值。
第四方面，本申请提供了一种计算装置，包括：
生成模块，用于针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；
发送模块，用于将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
可选地，发送模块被设置为：
按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
可选地，还包括：
配置模块，用于根据加速器中的计算核的资源配置确定并行数N的取值。
可选地，配置模块被设置为：
基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；
基于约束公式确定N的取值。
第五方面，本申请提供了一种电子设备，包括：
存储器，用于存储计算机程序；
处理器，用于执行计算机程序，以实现前述公开的计算方法。
第六方面，本申请提供了一种非易失性可读存储介质，用于保存计算机程序，其中，计算机程序被处理器执行时实现前述公开的计算方法。
第七方面，本申请提供了一种计算系统，包括：服务器以及至少一个前述公开的加速器。
通过以上方案可知，本申请提供了一种计算核，包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；第一缓存器件，用于根据接收到的计算指令从外部的第一存储器中加载参与图神经网络中的当前矩阵乘操作的第一矩阵中的数据；第二缓存器件，用于根据计算指令从外部的第二存储器中加载参与当前矩阵乘操作的第二矩阵中的数据；矩阵乘计算器件，用于按照计算指令设定的并行数N并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果；输出器件，用于暂存矩阵乘结果，在矩阵乘结果的数据量满足预设写条件时，将矩阵乘结果写入计算指令设定的第一存储器或第二存储器；其中，第二存储器的传输带宽大于第一存储器；若矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则计算指令设定有第二存储器；否则，计算指令设定有第一存储器。
可见，本申请提供的计算核能够并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果，因此可一次性得到N个最终矩阵乘结果，提高了计算效率和速度；计算核也无需暂存中间结果，节约了片上资源。在得到矩阵乘结果后，本申请能够按照该矩阵乘结果参与下一矩阵乘操作中的每一轮计算时的参与方式确定将该矩阵乘结果存入哪个存储器，可选的，当矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则将该矩阵乘结果存入第二存储器；否则，该矩阵乘结果存入第一存储器。由此可实现：使矩阵乘结果在存储器中的存储格式与其参与计算时的输出格式一致，便于在连续的计算过程中顺序读取数据，也无需进行矩阵转置，因此还能降低访存的时间 开销，提高效率。在满足预设写条件时，再写入矩阵乘结果，可降低写入次数，提升写入效率和带宽利用率。
相应地，本申请提供的一种加速器、计算方法、装置、设备、非易失性可读存储介质及系统，也同样具有上述技术效果。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请公开的一种计算核示意图；
图2为本申请公开的一种计算单元结构示意图；
图3为本申请公开的一种加速器结构示意图；
图4为本申请公开的另一种加速器结构示意图；
图5为本申请公开的第三种加速器结构示意图；
图6为本申请公开的一种计算方法流程图；
图7为本申请公开的一种数据并行输入示意图；
图8为本申请公开的一种4*4*4的PE阵列示意图；
图9本申请公开的一种写入数据连续存储示意图；
图10为本申请公开的一种矩阵乘计算流程示意图；
图11为本申请公开的一种电子设备示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
目前，现有的计算器件可以采用矩阵分块操作进行矩阵乘操作，但此方式需要将大量的中间结果暂存到计算器件之外，计算器件在进行后续计算时再读回至片内进行累加，增加了许多额外的数据搬移操作，会浪费计算资源，降低计算速度。为此，本申请提供了一种方案，能够提高图神经网络中矩阵乘操作的计算速度和计算效率。
参见图1所示，本申请实施例公开了一种计算核，包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件。
其中，第一缓存器件，被设置为根据接收到的计算指令从外部的第一存储器中加载参与图神经网络中的当前矩阵乘操作的第一矩阵中的数据；第二缓存器件，被设置为根据计算指令从外部的第二存储器中加 载参与当前矩阵乘操作的第二矩阵中的数据；矩阵乘计算器件，被设置为按照计算指令设定的并行数N并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果；输出器件，被设置为暂存矩阵乘结果，在矩阵乘结果的数据量满足预设写条件时，将矩阵乘结果写入计算指令设定的第一存储器或第二存储器。其中，一个行数据为：矩阵中的一行数据；一个列数据为：矩阵中的一列数据。
其中，第二存储器(如HBM)的传输带宽大于第一存储器(如DDR)；若矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则计算指令设定有第二存储器；否则，计算指令设定有第一存储器。HBM(High Bandwidth Memory)是一种适用于高带宽的CPU/GPU内存芯片。DDR(Double Data Rate，双倍速率同步动态随机存储器)相比与HBM，适用带宽较低。
其中，计算指令用于实现图神经网络中的任一矩阵乘操作，其中的并行数N的取值根据计算核的资源配置确定。可选的，基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；基于约束公式确定N的取值。
在一种可选的实施方式中，第一缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第一存储器中加载参与当前矩阵乘操作的第一矩阵中的剩余未加载数据；相应地，第二缓存器件被设置为：在矩阵乘计算器件计算矩阵乘结果的过程中，从第二存储器中加载参与当前矩阵乘操作的第二矩阵中的剩余未加载数据。由此便无需等待上一操作完成，就能提前进行数据加载，计算器件也不会出现空闲状态，能持续进行数据计算，提高了计算资源利用率和系统计算效率，降低任务延迟。
在一种可选的实施方式中，第一缓存器件利用乒乓方式加载第一矩阵中的数据。第一缓存器件可以为URAM(Ultra(超高效)RAM)。
在一种可选的实施方式中，第二缓存器件利用先入先出队列加载第二矩阵中的数据。可选的，第二缓存器件利用第一先入先出队列缓存第二矩阵中的数据，利用第二先入先出队列缓存矩阵乘结果。
在一种可选的实施方式中，矩阵乘计算器件包括：至少一个计算单元，计算单元利用乘法器和累加器构成且每次计算的数据量等于N。请参见图2，任一个计算单元可以包括至少一个乘法器组，一个乘法器组中包括两个乘法器；并且，一个乘法器组的输出连接累加器。如图2所示，每次计算的数据量为4，故有4个乘法器组。如有需要，乘法器组及其输出所连累加器可灵活增设或删除，由此可实现计算器件的灵活扩展和调整。图2中的量化截取用于实现：数据位宽的量化与截取，如果数据位宽为8，那么乘法结果的位宽就为16，再经过累加，数据位宽会更大，此时需要将最终结果的数据位宽截取为8位。
其中，每次计算的数据量等于N＝行并行数＝列并行数，可以使参与计算的行/列矩阵在内存的存储格式一致，便于图神经网络中每步计算结果的行优先/列优先转存(左右矩阵)变换，使得在连续的计算过程中可以顺序访存。
本实施例提供的计算核能够并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果，因此可一次性得到N个最终矩阵乘结果，提高了计算效率和速度；计算核也无需暂存中间结果，节约了片上资源。
本申请在得到矩阵乘结果后，能够按照该矩阵乘结果参与下一矩阵乘操作中的每一轮计算时的参与方式确定将该矩阵乘结果存入哪个存储器，可选的，当矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则将该矩阵乘结果存入第二存储器；否则，该矩阵乘结果存入第一存储器。由此可 实现：使矩阵乘结果在存储器中的存储格式与其参与计算时的输出格式一致，便于在连续的计算过程中顺序读取数据，也无需进行矩阵转置，因此还能降低访存的时间开销，提高效率。而在满足预设写条件时，再写入矩阵乘结果，可降低写入次数，提升写入效率和带宽利用率。可见，本实施例能够提高图神经网络中矩阵乘操作的计算速度和计算效率。
下面对本申请实施例提供的一种加速器进行介绍，下文描述的一种加速器与上文描述的一种计算核可以相互参照。
本申请实施例公开了一种加速器，包括：控制器、第一存储器、传输带宽大于第一存储器的第二存储器、以及前文的计算核。
其中，控制器与第一存储器、第二存储器、计算核分别连接；计算核与第一存储器、第二存储器分别连接。控制器，被设置为将图神经网络中的矩阵乘操作的计算指令发送至第一存储器、第二存储器和计算核；第一存储器，被设置为根据计算指令以直接内存访问DMA(Direct Memory Access)方式获取并存储参与当前矩阵乘操作的第一矩阵；第二存储器，被设置为根据计算指令以DMA方式获取并存储参与当前矩阵乘操作的第二矩阵；计算核，被设置为根据计算指令计算第一矩阵和第二矩阵的矩阵乘结果。
其中，第二存储器(如HBM)的传输带宽大于第一存储器(如DDR)，且以遍历方式参与计算的矩阵存入第二存储器，可使以遍历方式参与计算的矩阵的读取速度更快，能够提升计算速度。
其中，计算核包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；第一缓存器件能够在矩阵乘计算器件计算矩阵乘结果的过程中，从第一存储器中加载参与当前矩阵乘操作的第一矩阵中的剩余未加载数据；相应地，第二缓存器件能够在矩阵乘计算器件计算矩阵乘结果的过程中，从第二存储器中加载参与当前矩阵乘操作的第二矩阵中的剩余未加载数据。由此便无需等待上一操作完成，就能提前进行数据加载，计算器件也不会出现空闲状态，能持续进行数据计算，提高了计算资源利用率和系统计算效率，降低任务延迟。
参见图3所示，加速器中可设有多个计算核；相应地，需设置调度核来使控制器对这些计算核进行调度和控制。
在一种可选的实施方式中，控制器还被设置为：控制多个计算核执行同一矩阵乘操作。在一种可选的实施方式中，控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内计算第一矩阵中的相同行数据或相同列数据与第二矩阵中的不同列数据或不同行数据的矩阵乘结果。例如：在同一计算周期(同一时间段)内，计算核1计算第一矩阵中第一行与第二矩阵中第一列的乘法结果，计算核2计算第一矩阵中第一行与第二矩阵中第二列的乘法结果，由此可实现：第一矩阵中第一行数据在多个计算核上的复用，可避免数据的重复读取，也能够提升计算效率。如图3所示，S矩阵中的行数据1(第1行数据)被计算核1和计算核2复用。
在一种可选的实施方式中，控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内分别计算第一矩阵中的不同列数据或不同行数据与第二矩阵中的相同列数据或相同行数据的矩阵乘结果。例如：在同一计算周期(同一时间段)内，计算核1计算第一矩阵中第一行与第二矩阵中第一列的乘法结果，计算核2计算第一矩阵中第二行与第二矩阵中第一列的乘法结果，由此可实现：第二矩阵中第一行数据在多个计算核上的复用，可避免数据的重复读取，也能够提升计算效率。如图4所示，遍历矩阵被计算核1和计算核2复用。
在一种可选的实施方式中，控制器还被设置为：控制不同计算核在同一计算周期内执行不同矩阵乘操 作。例如：在同一计算周期(同一时间段)内，计算核1计算第一矩阵中第一行与第二矩阵中第一列的乘法结果，计算核2计算第三矩阵中第一行与第四矩阵中第一列的乘法结果，由此可实现：第一矩阵与第二矩阵的矩阵乘操作、第三矩阵与第四矩阵的矩阵乘操作同时执行，能够提高计算效率。如图5所示，计算核1计算矩阵A与矩阵B的乘法结果，计算核2计算矩阵C与矩阵D的乘法结果。
由于需要连续遍历整个遍历矩阵中的数据，因此可使用高带宽的HBM实现遍历矩阵的数据缓存；而使用单端口带宽低的DDR4缓存S矩阵中少量几行/列数据。使用高、低带宽的存储器除了分别给遍历矩阵和S矩阵提供合理的带宽之外，由于可在计算的同时继续加载后续数据，因此能够隐藏访存时间开销，提高系统整体吞吐效率。如图3所示，只需要加载S矩阵中少量几行/列数据于片上缓存，而后使用FIFO(First In First Out，先进先出)流式加载遍历矩阵，便可实现S矩阵中的数据复用。也即：S矩阵中的数据被多个计算核共用，既可以按照数据内存组织格式依次写回数据，又可避免共享外存接口的访问冲突。
在一种可选的实施方式中，计算指令用于实现图神经网络中的任意矩阵乘操作。图神经网络中的主要运算为：特征矩阵F与相应的小波矩阵Way、权重矩阵W相乘，用公式表示为：Way×(F×W)。如该公式所示，特征矩阵F在与W相乘时，F为乘号左边的左矩阵，而F×W的乘积与Way相乘时，该乘积为乘号右边的右矩阵，这种位置变换就需要将F×W的乘积按照该乘积在下一次乘法操作中的位置提前确定该乘积是行优先存储和列优先存储。当前一次矩阵乘结果需要存为行优先矩阵用于下一次计算时，前一次矩阵乘过程为：保持左矩阵中N行数据不动，遍历右矩阵的所有列，得到N行乘所有列的结果后；再次计算N+1～2N行乘所有列的结果，直至计算完成；此过程在按序生成行优先矩阵的同时，对左矩阵的行数据进行了数据重用。当前一次矩阵乘结果需要存为列优先矩阵用于下一次计算时，前一次矩阵乘过程为：保持右矩阵中N列数据不动，遍历左矩阵的所有行，得到N列乘所有行的结果后；再次计算N+1～2N列乘所有行的结果，直至计算完成；此过程在按序生成列优先矩阵的同时，对右矩阵的列数据进行了数据重用。
其中，任一个计算核包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；矩阵乘计算器件包括：至少一个计算单元，该计算单元利用乘法器和累加器构成且每次计算的数据量等于N。每次计算的数据量等于N＝行并行数＝列并行数，可以使参与计算的行/列矩阵在内存的存储格式一致，便于图神经网络中每步计算结果的行优先/列优先转存(左右矩阵)变换，使得在连续的计算过程中可以顺序访存。通过灵活设置N的取值可调整计算加速比。可选的，并行数N的取值根据计算核的资源配置确定。例如：基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；基于约束公式确定N的取值。
可见，本实施例提供了一种加速器，利用该加速器能够避免矩阵转置的访存和时间开销，实现计算过程中的数据重用，提高图神经网络中矩阵乘操作的计算速度和计算效率，还可实现计算器件的灵活并行扩展。
下面对本申请实施例提供的一种计算方法进行介绍，下文描述的一种计算方法与其他实施例可以相互参照。
参见图6所示，本申请实施例公开了一种计算方法，包括：
S601、针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置。
S602、将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
在图神经网络中有很多节点，每一节点对应有一个特征矩阵。特征矩阵F与相应的小波矩阵Way、权重矩阵W相乘，用公式表示为：Way×(F×W)。图神经网络中包括多个这样的乘法步骤，故需要按照图神经网络中不同矩阵乘操作的计算顺序确定相应计算指令的顺序，而后按序发送计算指令至加速器，以使加速器按序进行图神经网络中的矩阵乘操作。在一种可选的实施方式中，将计算指令发送至前文的加速器，包括：按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
在一种可选的实施方式中，根据加速器中的计算核的资源配置确定并行数N的取值。在一种可选的实施方式中，根据加速器中的计算核的资源配置确定并行数N的取值，包括：基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；基于约束公式确定N的取值。可选的，求解满足Count≥N3/2、BW≥DW×f×N2的最大N值作为并行数。
可见，本实施例提供了一种计算方法，能够避免矩阵转置的访存和时间开销，实现计算过程中的数据重用，提高图神经网络中矩阵乘操作的计算速度和计算效率，还可实现计算器件的灵活并行扩展。
下面对本申请实施例提供的一种计算装置进行介绍，下文描述的一种计算装置与其他实施例可以相互参照。
本申请实施例公开了一种计算装置，包括：
生成模块，被设置为针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；
发送模块，被设置为将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
在一种可选的实施方式中，发送模块被设置为：
按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
在一种可选的实施方式中，还包括：
配置模块，被设置为根据加速器中的计算核的资源配置确定并行数N的取值。
在一种可选的实施方式中，配置模块被设置为：
基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；
基于约束公式确定N的取值。
其中，关于本实施例中各个模块、单元更加详细的工作过程可以参考其他实施例中公开的相应内容，在此不再进行赘述。
可见，本实施例提供了一种计算装置，能够避免矩阵转置的访存和时间开销，实现计算过程中的数据重用，提高图神经网络中矩阵乘操作的计算速度和计算效率，还可实现计算器件的灵活并行扩展。
基于上述任意实施例，下面提供一种用于图神经网络中矩阵乘操作的计算器件的设计方案。如图7所示，通过3个参数kernel长度、行并行数R、列并行数C设计构成计算器件的PE(Process Element，计算单元)阵列，且kernel长度＝行并行数R＝列并行数C。在图7中，矩阵中的一行数据或一列数据被划分为7个K大小的数据块，图7所示的PE阵列可同时计算输入给其的数据。也就是说，当kernel长度＝行并行数R＝列并行数C＝8时，同时输入8行数据和8列数据至PE阵列，使得PE阵列同时计算8行数据和8列数据的乘法结果。需要说明的是，图7示意的7个K大小的数据块仅为示例，在实际实现时，数据块的个数应为2的指数，即：一行数据或一列数据被划分为2的指数个K大小的数据块。
在本实施例中，kernel长度、行并行数R、列并行数C根据单一PE的DSP资源和访存带宽来设置；其中，DSP使用数量为：CountDSP＝K*R*(C/2)；K为kernel长度，R为行并行数，C为列并行数；其访存带宽为：BWtraverse＝DW*K*C*f；DW为每个数据的位宽，K为kernel长度，C为列并行数，f为DSP运行时钟频率。例如，数据为int8，K＝C＝16，f为400M/s时，BWtraverse＝8bit*16*16*400M/s＝100GB/s。由此可确定并行数的取值。由于kernel长度＝行并行数R＝列并行数C，因此在实际实现时，上述公式可用Count≥N3/2、BW≥DW×f×N2代替，N＝K＝R＝C。
在本实施例中，kernel为参与计算的最小单位，单一PE采用乘法器和加法器结构，该结构的好处是：能够并行加速计算过程。如果行的长度为L，那么整行/整列顺序计算的时间为L，而使用图2所示结构，用时为log2(L)。
在一种示例中，4*4*4的PE阵列如图8所示。在图8中，每个PE中都存在一组数据的复用。
其中，每次计算的数据量等于N＝行并行数＝列并行数，可以使参与计算的行/列矩阵在内存的存储格式一致，便于图神经网络中每步计算结果的行优先/列优先转存(左右矩阵)变换，使得在连续的计算过程中可以顺序访存。例如：4行×4列的A矩阵和4行×N列的B矩阵相乘时，在R＝C＝4时，针对A矩阵的1-4行和B矩阵的1-4列数据，每时钟周期取K个数据，经过完整行/列遍历后，得到结果矩阵中1-4行1-4列的数据；然后再取B矩阵5-8列数据，和A矩阵1-4行数据再次遍历后，得到结果矩阵中1-4行5-8列的数据，如此循环，直到结果矩阵1-4行数据计算完成，再切换A矩阵5-8行数据继续进行遍历；这样结果矩阵就和A矩阵的数据排列一致，从而保证每次矩阵计算时输入、输出矩阵格式的一致性。本例属于行优先矩阵，列优先矩阵类似。可选的，结果矩阵的存储顺序如图9所示。
本实施例提供的计算核能够并行计算第一矩阵中的N个行数据或N个列数据与第二矩阵中的N个列数据或N个行数据的矩阵乘结果，因此可一次性得到N个最终矩阵乘结果，提高了计算效率和速度；计算核也无需暂存中间结果，节约了片上资源。
请参见图10，按照本实施例实现的矩阵乘操作流程包括：
(1)执行重用矩阵加载指令，从DDR读取stationary矩阵(S矩阵，重用矩阵，即该矩阵相对于遍历矩阵来说，数据可重复使用)的一组行/列数据到URAM，由于URAM的存储深度通常较大，可以在当前组计算过程中提前将下一组数据读入进行乒乓缓存；
(2)执行遍历矩阵加载指令，从HBM依次读取遍历矩阵的所有列/行数据到FIFO；
(3)执行矩阵乘计算指令，PE阵列进行计算；
(4)计算完成一组数据后，执行结果写回指令，根据指令中定义的地址通过总线burst(突发)传输将结果写到对应的外部HBM/DDR4；
(5)复位PE阵列中的累加器，以便进行下一次计算；
重复(1)-(4)步，直到S矩阵所有组数据计算完成。
其中，通过总线burst传输将结果写到对应的外部HBM/DDR4，可以减少写入次数。也即：凑够一个burst长度，向外部存储器写一次。例如：如果当前加速器为基于FPGA的系统，其中所用的AXI4(Advanced eXtensible Interface 4，一种总线协议)的burst长度为4KB，而对于int8数据类型，需要凑够4096个数据向部存储器写一次。
可见，本实施例使用定量设计方法，根据片上资源及外部可提供的访存带宽，得出PE阵列的组织形式及规模；使用行/列优先矩阵的生成方法生成图神经网络下一步计算所需格式的特征矩阵；使用完整的硬件 加速系统及方法进行数据加载、遍历计算及结果写回；可灵活扩展的多PE阵列系统架构用于单个或多个矩阵乘的并行计算加速；将操作指令拆分为数据加载、计算和结果写回指令，提前进行数据加载，保证PE阵列连续不间断工作。该方案可灵活适配硬件资源，实现指令周期性流水处理，有效提升计算和访存效率，完成图神经网络中各级矩阵的灵活变换和连续高效计算。
下面对本申请实施例提供的一种电子设备进行介绍，下文描述的一种电子设备与其他实施例可以相互参照。
参见图11所示，本申请实施例公开了一种电子设备，包括：
存储器1101，被设置为保存计算机程序；
处理器1102，被设置为执行计算机程序，以实现上述任意实施例公开的方法。
在本实施例中，处理器执行存储器中保存的计算机程序时，可以实现以下步骤：针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
在本实施例中，处理器执行存储器中保存的计算机程序时，可以实现以下步骤：按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
在本实施例中，处理器执行存储器中保存的计算机程序时，可以实现以下步骤：根据加速器中的计算核的资源配置确定并行数N的取值。
在本实施例中，处理器执行存储器中保存的计算机程序时，可以实现以下步骤：基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；基于约束公式确定N的取值。
可选的，本申请实施例还提供了一种服务器来作为上述电子设备。该服务器，可以包括：至少一个处理器、至少一个存储器、电源、通信接口、输入输出接口和通信总线。其中，存储器被设置为存储计算机程序，计算机程序由处理器加载并执行，以实现前述任一实施例公开的计算方法中的相关步骤。
本实施例中，电源被设置为服务器上的各硬件设备提供工作电压；通信接口能够为服务器创建与外界设备之间的数据传输通道，其所遵循的通信协议是能够适用于本申请技术方案的任意通信协议，在此不对其进行限定；输入输出接口，被设置为获取外界输入数据或向外界输出数据，其接口类型可以根据应用需要进行选取，在此不进行限定。
另外，存储器作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，其上所存储的资源包括操作系统、计算机程序及数据等，存储方式可以是短暂存储或者永久存储。
其中，操作系统被设置为管理与控制服务器上的各硬件设备以及计算机程序，以实现处理器对存储器中数据的运算与处理，其可以是Windows Server、Netware、Unix、Linux等。计算机程序除了包括能够被设置为完成前述任一实施例公开的计算方法的计算机程序之外，还可以包括能够被设置为完成其他特定工作的计算机程序。数据除了可以包括虚拟机等数据外，还可以包括虚拟机的开发商信息等数据。
可选的，本申请实施例还提供了一种终端来作为上述电子设备。该终端可以包括但不限于智能手机、平板电脑、笔记本电脑或台式电脑等。
通常，本实施例中的终端包括有：处理器和存储器。
其中，处理器可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field-Programmable Gate Array，现场可编程 门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器也可以包括主处理器和协处理器，主处理器是被设置为对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是被设置为对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU被设置为负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器被设置为处理有关机器学习的计算操作。
存储器可以包括一个或多个非易失性可读存储介质，该非易失性可读存储介质可以是非暂态的。存储器还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。本实施例中，存储器至少被设置为存储以下计算机程序，其中，该计算机程序被处理器加载并执行之后，能够实现前述任一实施例公开的由终端侧执行的计算方法中的相关步骤。另外，存储器所存储的资源还可以包括操作系统和数据等，存储方式可以是短暂存储或者永久存储。其中，操作系统可以包括Windows、Unix、Linux等。数据可以包括但不限于应用程序的更新信息。
在一些实施例中，终端还可包括有显示屏、输入输出接口、通信接口、传感器、电源以及通信总线。
下面对本申请实施例提供的一种非易失性可读存储介质进行介绍，下文描述的一种非易失性可读存储介质与其他实施例可以相互参照。
本申请实施例公开了一种非易失性可读存储介质，被设置为保存计算机程序，其中，计算机程序被处理器执行时实现前述实施例公开的计算方法。其中，非易失性可读存储介质作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，其上所存储的资源包括操作系统、计算机程序及数据等，存储方式可以是短暂存储或者永久存储。
在本实施例中，处理器执行的计算机程序，可以实现以下步骤：针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；将计算指令发送至前文的加速器，以使加速器对图神经网络进行加速计算。
在本实施例中，处理器执行的计算机程序，可以实现以下步骤：按照图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至加速器。
在本实施例中，处理器执行的计算机程序，可以实现以下步骤：根据加速器中的计算核的资源配置确定并行数N的取值。
在本实施例中，处理器执行的计算机程序，可以实现以下步骤：基于计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示计算核的计算资源数量，BW表示计算核的访存带宽，DW表示计算核的数据位宽，f为计算核的计算时钟频率；基于约束公式确定N的取值。
下面对本申请实施例提供的一种计算系统进行介绍，下文描述的一种计算系统与其他实施例可以相互参照。
本申请实施例公开了一种计算系统，包括：服务器以及至少一个前述公开的加速器。服务器用于针对图神经网络中的每一矩阵乘操作生成计算指令；计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；将计算指令发送至任一个加速器。接收到计算指令的加速器对图神经网络进行加速计算。
可见，本实施例提供了一种计算系统，能够避免矩阵转置的访存和时间开销，实现计算过程中的数据重用，提高图神经网络中矩阵乘操作的计算速度和计算效率，还可实现计算器件的灵活并行扩展。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处， 各个实施例之间相同或相似部分互相参见即可。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的非易失性可读存储介质中。
本文中应用了可选的个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在可选的实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (20)

  1. 一种计算核，其特征在于，包括：第一缓存器件、第二缓存器件、矩阵乘计算器件和输出器件；
    所述第一缓存器件，被设置为根据接收到的计算指令从外部的第一存储器中加载参与图神经网络中的当前矩阵乘操作的第一矩阵中的数据；
    所述第二缓存器件，被设置为根据所述计算指令从外部的第二存储器中加载参与当前矩阵乘操作的第二矩阵中的数据；
    所述矩阵乘计算器件，被设置为按照所述计算指令设定的并行数N并行计算所述第一矩阵中的N个行数据或N个列数据与所述第二矩阵中的N个列数据或N个行数据的矩阵乘结果；
    所述输出器件，被设置为暂存所述矩阵乘结果，在所述矩阵乘结果的数据量满足预设写条件时，将所述矩阵乘结果写入所述计算指令设定的所述第一存储器或所述第二存储器；
    其中，所述第二存储器的传输带宽大于所述第一存储器；若所述矩阵乘结果参与下一矩阵乘操作中的每一轮计算时需以遍历方式参与计算，则所述计算指令设定有所述第二存储器；否则，所述计算指令设定有所述第一存储器。
  2. 根据权利要求1所述的计算核，其特征在于，所述第一缓存器件被设置为：在所述矩阵乘计算器件计算所述矩阵乘结果的过程中，从所述第一存储器中加载参与当前矩阵乘操作的第一矩阵中的剩余未加载数据；
    相应地，所述第二缓存器件被设置为：在所述矩阵乘计算器件计算所述矩阵乘结果的过程中，从所述第二存储器中加载参与当前矩阵乘操作的第二矩阵中的剩余未加载数据。
  3. 根据权利要求1所述的计算核，其特征在于，所述第一缓存器件利用乒乓方式加载所述第一矩阵中的数据。
  4. 根据权利要求1所述的计算核，其特征在于，所述第二缓存器件利用先入先出队列加载所述第二矩阵中的数据。
  5. 根据权利要求1至4任一项所述的计算核，其特征在于，所述矩阵乘计算器件包括：至少一个计算单元，所述计算单元利用乘法器和累加器构成且每次计算的数据量等于N。
  6. 一种加速器，其特征在于，包括：控制器、第一存储器、传输带宽大于所述第一存储器的第二存储器、以及如权利要求1至5任一项所述的计算核；
    其中，所述控制器与所述第一存储器、所述第二存储器、所述计算核分别连接；所述计算核与所述第一存储器、所述第二存储器分别连接；
    所述控制器，被设置为将图神经网络中的矩阵乘操作的计算指令发送至所述第一存储器、所述第二存储器和所述计算核；
    所述第一存储器，被设置为根据所述计算指令以直接内存访问DMA方式获取并存储参与当前矩阵乘操作的第一矩阵；
    所述第二存储器，被设置为根据所述计算指令以DMA方式获取并存储参与当前矩阵乘操作的第二矩阵；
    所述计算核，被设置为根据所述计算指令计算所述第一矩阵和所述第二矩阵的矩阵乘结果。
  7. 根据权利要求6所述的加速器，其特征在于，所述计算核有多个。
  8. 根据权利要求7所述的加速器，其特征在于，所述控制器还被设置为：控制多个计算核执行同一矩阵乘操作。
  9. 根据权利要求8所述的加速器，其特征在于，所述控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内计算所述第一矩阵中的相同行数据或相同列数据与所述第二矩阵中的不同列数据或不同行数据的矩阵乘结果。
  10. 根据权利要求8所述的加速器，其特征在于，所述控制器被设置为：在控制多个计算核执行同一矩阵乘操作时，使每一计算核在同一计算周期内分别计算所述第一矩阵中的不同列数据或不同行数据与所述第二矩阵中的相同列数据或相同行数据的矩阵乘结果。
  11. 根据权利要求7所述的加速器，其特征在于，所述控制器还被设置为：控制不同计算核在同一计算周期内执行不同矩阵乘操作。
  12. 根据权利要求6所述的加速器，其特征在于，所述计算指令用于实现图神经网络中的任意矩阵乘操作。
  13. 一种计算方法，其特征在于，包括：
    针对图神经网络中的每一矩阵乘操作生成计算指令；所述计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；
    将所述计算指令发送至加速器，以使所述加速器对所述图神经网络进行加速计算；所述加速器为权利要求6至12任一项所述的加速器。
  14. 根据权利要求13所述的方法，其特征在于，所述将所述计算指令发送至加速器，包括：
    按照所述图神经网络中不同矩阵乘操作的计算顺序将相应计算指令按序发送至所述加速器。
  15. 根据权利要求13所述的方法，其特征在于，还包括：
    根据所述加速器中的计算核的资源配置确定并行数N的取值。
  16. 根据权利要求15所述的方法，其特征在于，所述根据所述加速器中的计算核的资源配置确定并行数N的取值，包括：
    基于所述计算核的资源配置构建约束公式：Count≥N3/2、BW≥DW×f×N2；其中，Count表示所述计算核的计算资源数量，BW表示所述计算核的访存带宽，DW表示所述计算核的数据位宽，f为所述计算核的计算时钟频率；
    基于所述约束公式确定N的取值。
  17. 一种计算装置，其特征在于，包括：
    生成模块，被设置为针对图神经网络中的每一矩阵乘操作生成计算指令；所述计算指令中设有并行数N以及相应矩阵乘操作的矩阵乘结果的写入位置；
    发送模块，被设置为将所述计算指令发送至加速器，以使所述加速器对所述图神经网络进行加速计算；所述加速器为权利要求6至12任一项所述的加速器。
  18. 一种电子设备，其特征在于，包括：
    存储器，被设置为存储计算机程序；
    处理器，被设置为执行所述计算机程序，以实现如权利要求13至16任一项所述的方法。
  19. 一种非易失性可读存储介质，其特征在于，被设置为保存计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求13至16任一项所述的方法。
  20. 一种计算系统，其特征在于，包括：服务器以及至少一个如权利要求6至12任一项所述的加速器。
PCT/CN2023/132743 2023-02-15 2023-11-20 计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统 Ceased WO2024169293A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US19/154,296 US12608319B1 (en) 2023-02-15 2023-11-20 Computing core, accelerator, computing method and apparatus, device, non-volatile readable storage medium, and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310114640.4 2023-02-15
CN202310114640.4A CN115860080B (zh) 2023-02-15 2023-02-15 计算核、加速器、计算方法、装置、设备、介质及系统

Publications (1)

Publication Number Publication Date
WO2024169293A1 true WO2024169293A1 (zh) 2024-08-22

Family

ID=85658058

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/132743 Ceased WO2024169293A1 (zh) 2023-02-15 2023-11-20 计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统

Country Status (3)

Country Link
US (1) US12608319B1 (zh)
CN (1) CN115860080B (zh)
WO (1) WO2024169293A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116346811A (zh) * 2023-03-29 2023-06-27 苏州浪潮智能科技有限公司 数据转存方法及装置

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115860080B (zh) * 2023-02-15 2023-05-09 苏州浪潮智能科技有限公司 计算核、加速器、计算方法、装置、设备、介质及系统
CN117290289B (zh) * 2023-11-27 2024-01-26 深存科技(无锡)有限公司 基于通用型cpu的矩阵加速器架构
US12045309B1 (en) * 2023-11-29 2024-07-23 Recogni Inc. Systems and methods for performing matrix multiplication with a plurality of processing elements
US12008069B1 (en) * 2023-11-29 2024-06-11 Recogni Inc. Multi-mode architecture for unifying matrix multiplication, 1×1 convolution and 3×3 convolution
CN118536073B (zh) * 2024-07-23 2024-09-20 浪潮(北京)电子信息产业有限公司 加速器、数据处理方法、设备、介质、程序产品及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106445471A (zh) * 2016-10-13 2017-02-22 北京百度网讯科技有限公司 处理器和用于在处理器上执行矩阵乘运算的方法
CN109522125A (zh) * 2018-11-19 2019-03-26 郑州云海信息技术有限公司 一种矩阵乘积转置的加速方法、装置及处理器
CN111967582A (zh) * 2020-08-07 2020-11-20 苏州浪潮智能科技有限公司 一种cnn卷积层运算方法及cnn卷积层运算加速器
CN113918120A (zh) * 2021-10-19 2022-01-11 Oppo广东移动通信有限公司 计算装置、神经网络处理设备、芯片及处理数据的方法
CN113947200A (zh) * 2021-12-21 2022-01-18 珠海普林芯驰科技有限公司 神经网络的加速计算方法及加速器、计算机可读存储介质
US20220188613A1 (en) * 2020-12-15 2022-06-16 The George Washington University Sgcnax: a scalable graph convolutional neural network accelerator with workload balancing
CN115860080A (zh) * 2023-02-15 2023-03-28 苏州浪潮智能科技有限公司 计算核、加速器、计算方法、装置、设备、介质及系统

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10997275B2 (en) * 2016-03-23 2021-05-04 Gsi Technology Inc. In memory matrix multiplication and its usage in neural networks
CN109213962B (zh) * 2017-07-07 2020-10-09 华为技术有限公司 运算加速器
US11861484B2 (en) * 2018-09-28 2024-01-02 Qualcomm Incorporated Neural processing unit (NPU) direct memory access (NDMA) hardware pre-processing and post-processing
US20220012304A1 (en) * 2020-07-07 2022-01-13 Sudarshan Kumar Fast matrix multiplication
GB2626214B (en) * 2021-06-29 2026-03-18 Imagination Tech Ltd Neural network comprising matrix multiplication
US20250103292A1 (en) * 2023-09-27 2025-03-27 Apple Inc. Matrix Multiplier Caching
US20260023818A1 (en) * 2025-09-26 2026-01-22 Altera Corporation Systems and Methods for a Near Memory-Based Matrix Computation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106445471A (zh) * 2016-10-13 2017-02-22 北京百度网讯科技有限公司 处理器和用于在处理器上执行矩阵乘运算的方法
CN109522125A (zh) * 2018-11-19 2019-03-26 郑州云海信息技术有限公司 一种矩阵乘积转置的加速方法、装置及处理器
CN111967582A (zh) * 2020-08-07 2020-11-20 苏州浪潮智能科技有限公司 一种cnn卷积层运算方法及cnn卷积层运算加速器
US20220188613A1 (en) * 2020-12-15 2022-06-16 The George Washington University Sgcnax: a scalable graph convolutional neural network accelerator with workload balancing
CN113918120A (zh) * 2021-10-19 2022-01-11 Oppo广东移动通信有限公司 计算装置、神经网络处理设备、芯片及处理数据的方法
CN113947200A (zh) * 2021-12-21 2022-01-18 珠海普林芯驰科技有限公司 神经网络的加速计算方法及加速器、计算机可读存储介质
CN115860080A (zh) * 2023-02-15 2023-03-28 苏州浪潮智能科技有限公司 计算核、加速器、计算方法、装置、设备、介质及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116346811A (zh) * 2023-03-29 2023-06-27 苏州浪潮智能科技有限公司 数据转存方法及装置
CN116346811B (zh) * 2023-03-29 2026-01-30 苏州元脑智能科技有限公司 数据转存方法及装置

Also Published As

Publication number Publication date
US20260111365A1 (en) 2026-04-23
CN115860080B (zh) 2023-05-09
CN115860080A (zh) 2023-03-28
US12608319B1 (en) 2026-04-21

Similar Documents

Publication Publication Date Title
WO2024169293A1 (zh) 计算核、加速器、计算方法、装置、设备、非易失性可读存储介质及系统
US6341318B1 (en) DMA data streaming
US20240070223A1 (en) Increased computation efficiency with multi-stage 8-bit floating point matrix multiplication with format conversion
CN111583095A (zh) 图像数据存储方法、图像数据处理方法、系统及相关装置
CN115880132B (zh) 图形处理器、矩阵乘法任务处理方法、装置及存储介质
CN102508803A (zh) 一种矩阵转置存储控制器
CN115983348B (zh) 支持卷积神经网络扩展指令的risc-v加速器系统
CN116710912A (zh) 一种矩阵乘法器及矩阵乘法器的控制方法
CN108897716A (zh) 通过存储器读写操作来缩减计算量的数据处理装置及方法
CN212112470U (zh) 一种矩阵乘法计算电路
CN117973455A (zh) 基于fpga的卷积神经网络加速器实现方法
CN116402102A (zh) 神经网络处理器和电子设备
CN116107959A (zh) 缓存方法、图像传输方法、电子设备及存储介质
WO2024027140A1 (zh) 一种数据处理方法、装置、设备、系统及可读存储介质
CN107678781A (zh) 处理器以及用于在处理器上执行指令的方法
CN110377874A (zh) 卷积运算方法及系统
CN120011299B (zh) 并行处理方法和装置
CN112558861B (zh) 一种面向多核处理器阵列的数据加载和存储系统及方法
WO2021179289A1 (zh) 卷积神经网络的运算方法、装置、设备和存储介质
CN118886500A (zh) 模型推理方法、装置、系统、存储介质及程序产品
EP4581483A1 (en) Increased computation efficiency with multi-stage 8-bit floating point matrix multiplication with format conversion
CN117408318A (zh) 一种基于fpga卷积神经网络的数据高效存取方法
CN116166185A (zh) 缓存方法、图像传输方法、电子设备及存储介质
CN121350398B (zh) 一种数据处理方法、装置、设备以及存储介质
CN118839097B (zh) 矩阵转置单元、计算装置、矩阵转置方法及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23922409

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23922409

Country of ref document: EP

Kind code of ref document: A1