WO2019206162A1 - 计算装置和计算方法 - Google Patents
计算装置和计算方法 Download PDFInfo
- Publication number
- WO2019206162A1 WO2019206162A1 PCT/CN2019/084011 CN2019084011W WO2019206162A1 WO 2019206162 A1 WO2019206162 A1 WO 2019206162A1 CN 2019084011 W CN2019084011 W CN 2019084011W WO 2019206162 A1 WO2019206162 A1 WO 2019206162A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- register
- convolution
- convolution operation
- register sets
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Definitions
- the present application relates to the field of data processing, and in particular, to a computing device and a computing method.
- Convolution operations are the most common computational operations in convolutional neural networks. Convolution operations are primarily applied to convolutional layers. In the convolution operation, each weight data in the convolution kernel is multiplied by its corresponding input data, and then the point multiplication results are added to obtain an output result of one convolution operation. Then, according to the step size setting of the convolution layer, the convolution kernel is slid and the above convolution operation is repeated. Convolutional neural networks typically process large amounts of data, and high data throughput requires high access bandwidth.
- a computing device for convolution operations typically includes a plurality of arithmetic units and register files that process data in parallel, the register file being used to store data to be subjected to convolution operations.
- each convolution operation multiple arithmetic units need to read the data required for the convolution operation from the register file. Since the amount of data processed by the convolution operation is usually very large, the operations of multiple arithmetic units in the convolution operation I need to read a lot of data from the register file.
- the application provides a computing device and a computing method, which can improve data utilization.
- a computing device comprising: a plurality of arithmetic units for performing a plurality of convolution operations on a plurality of data, the plurality of arithmetic units including a first arithmetic unit and a second arithmetic unit;
- the first operation unit is configured to: in the ith convolution operation of the multiple convolution operation, multiply the first weight data and the first data of the plurality of data, where i is An integer greater than 0; receiving second data transmitted by the second operation unit, wherein the second data is data of a second operation unit performing a convolution operation in the ith convolution operation; In the i+1th convolution operation in the multiple convolution operation, the first weight data and the second data are multiplied.
- the first operation unit may acquire the data repeated in the two convolution operations from the second operation unit, without reading the repeated data to be convolved from the register file, thereby improving the utilization of the data.
- the rate reduces the amount of data exchanged between the arithmetic unit and the register file, reducing the overhead of access bandwidth between the arithmetic unit and the register file.
- the method further includes: a plurality of register sets for storing the plurality of data; the plurality of operation units are further configured to acquire the multiple convolutions from the plurality of register sets The plurality of data required in operation, wherein each of the plurality of arithmetic units is capable of acquiring data stored in any one of the plurality of register sets.
- any one of the plurality of operation units can access any of the register groups, so the plurality of register groups only need to read in and store the data to be convolved once, without having to read from the memory multiple times and The large amount of data repeated in the multiple convolution operation is stored, thereby saving the storage space of a plurality of register banks, and reducing the pressure of access bandwidth between the plurality of register banks and the memory, thereby improving the parallelism of data access.
- the first operation unit is configured to: acquire the first data from a first register group of the plurality of register groups in the ith convolution operation
- the second arithmetic unit is configured to: acquire, in the i-th convolution operation, the second data from a second one of the plurality of register sets, the first register set and The second register set is a different register set.
- data read by the plurality of operation units from the plurality of register sets is located in different register groups of the plurality of register sets. in.
- a flexible data arrangement manner is adopted in a plurality of register groups, so that as much data as possible can be read out in one clock cycle, data read conflicts are avoided, and a non-blocking convolution operation process is realized.
- the convolution operation corresponds to a convolution kernel size of K*K
- the plurality of register sets includes K*K register sets
- the plurality of data includes M rows and N columns of data.
- the distribution of the plurality of data in the K*K register sets meets the condition that data in the xth row and the sth column of the plurality of data is stored in the bth register group,
- the data in the xth row and the sth column of the data are stored in the [b+(a*K)] mod(K*K) register group, where K is an integer not less than 1, and mod is performed.
- M ⁇ x ⁇ 1, N ⁇ s ⁇ 1, K*K ⁇ b ⁇ 1, M,N,x,s,b,a are positive integers.
- a computing method including: a computing device applying the computing method includes a plurality of computing units, the plurality of computing units configured to perform a plurality of convolution operations on a plurality of data, the plurality of The operation unit includes a first operation unit and a second operation unit; the method includes: in the i-th convolution operation of the multiple convolution operation, the first operation unit pairs the first weight data and the The first data of the plurality of data is multiplied, wherein i is an integer greater than 0; the first operation unit receives the second data sent by the second operation unit, wherein the second data is The second arithmetic unit performs data of a convolution operation in the i-th convolution operation; in the i+1th convolution operation in the multiple convolution operation, the first operation unit The first weight data and the second data are multiplied.
- the first operation unit may acquire the data repeated in the two convolution operations from the second operation unit, without reading the repeated data to be convolved from the register file, thereby improving the utilization of the data.
- the rate reduces the amount of data exchanged between the arithmetic unit and the register file, reducing the overhead of access bandwidth between the arithmetic unit and the register file.
- the computing device further includes: a plurality of register sets for storing the plurality of data; wherein each of the plurality of operation units is capable of acquiring Data stored in any one of a plurality of register sets.
- any one of the plurality of operation units can access any of the register groups, so the plurality of register groups only need to read in and store the data to be convolved once, without having to read from the memory multiple times and The large amount of data repeated in the multiple convolution operation is stored, thereby saving the storage space of a plurality of register banks, and reducing the pressure of access bandwidth between the plurality of register banks and the memory, thereby improving the parallelism of data access.
- the method further includes: in the ith convolution operation, the first operation unit acquires the first data from a first register group of the plurality of register sets; In the ith convolution operation, the second arithmetic unit acquires the second data from a second one of the plurality of register sets, the first register set and the second register Groups are different register sets.
- data read by the plurality of operation units from the plurality of register sets is located in different register groups of the plurality of register sets. in.
- a flexible data arrangement manner is adopted in a plurality of register groups, so that as much data as possible can be read out in one clock cycle, data read conflicts are avoided, and a non-blocking convolution operation process is realized.
- the convolution operation corresponds to a convolution kernel size of K*K
- the plurality of register sets includes K*K register sets
- the plurality of data includes M rows and N columns of data.
- the distribution of the plurality of data in the K*K register sets meets the condition that data in the xth row and the sth column of the plurality of data is stored in the rth group register, the plurality of The data in the xth row and the sth column of the data is stored in the [r+(a*K)] mod(K*K) group register, where K is an integer not less than 1, and mod indicates that the remainder is performed.
- M ⁇ x ⁇ 1, N ⁇ s ⁇ 1, K * K ⁇ r ⁇ 1, M, N, x, s, r, a is a positive integer.
- a computing device comprising: a plurality of operation units and a plurality of register sets, wherein each of the operation units is respectively connected to the plurality of register sets; the plurality of register sets are used to store a plurality of register sets to be executed a plurality of data of a sub-convolution operation; the plurality of operation units, configured to acquire the plurality of data from the plurality of register sets, and perform a plurality of convolution operations on the plurality of data, wherein Each of the plurality of arithmetic units is configured to acquire data stored in any one of the plurality of register sets.
- any one of the plurality of operation units can access any of the register groups, so the plurality of register groups only need to read in and store the data to be convolved once, without having to read from the memory multiple times and The large amount of data repeated in the multiple convolution operation is stored, thereby saving the storage space of a plurality of register banks, and reducing the pressure of access bandwidth between the plurality of register banks and the memory, thereby improving the parallelism of data access.
- the data read by the plurality of operation units from the plurality of register sets is located in the multiple In a different register set of register banks, where i is an integer greater than zero.
- a flexible data arrangement manner is adopted in a plurality of register groups, so that as much data as possible can be read out in one clock cycle, data read conflicts are avoided, and a non-blocking convolution operation process is realized.
- the convolution operation corresponds to a convolution kernel size of K*K
- the plurality of register sets includes K*K register sets
- the plurality of data includes M rows and N columns of data.
- the distribution of the plurality of data in the K*K register sets meets the condition that data in the xth row and the sth column of the plurality of data is stored in the bth register group,
- the data in the xth row and the sth column of the data are stored in the [b+(a*K)] mod(K*K) register group, where K is an integer not less than 1, and mod is performed.
- M ⁇ x ⁇ 1, N ⁇ s ⁇ 1, K*K ⁇ b ⁇ 1, M,N,x,s,b,a are positive integers.
- the multiple operation units include a first operation unit and a second operation unit
- the first operation unit is configured to: the ith convolution in the multiple convolution operation In operation, multiplying the first weight data and the first data of the plurality of data, wherein i is an integer greater than 0; receiving second data sent by the second computing unit, wherein The second data is data in which the second arithmetic unit performs a convolution operation in the i-th convolution operation; in the i+1th convolution operation in the multiple convolution operation, The first weight data and the second data are multiplied.
- the first operation unit may acquire the data repeated in the two convolution operations from the second operation unit, without reading the repeated data to be convolved from the register file, thereby improving the utilization of the data.
- the rate reduces the amount of data exchanged between the arithmetic unit and the register file, reducing the overhead of access bandwidth between the arithmetic unit and the register file.
- a fourth aspect a chip is provided, wherein the chip is provided with the computing device as described in the first aspect or the possible implementation manner of any of the first aspects, or the chip is provided with the third aspect or A computing device as described in any one of the possible implementations of the third aspect.
- a computer storage medium the program storing program code for execution by a computing device, the program code comprising instructions for performing the method of the second aspect.
- FIG. 1 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
- FIG. 2 is a schematic diagram of a convolution operation of convolutional data of different periods and data to be convolved.
- FIG. 3 is a schematic structural diagram of a computing device according to an embodiment of the present application.
- FIG. 4 is a schematic structural diagram of a computing device according to still another embodiment of the present application.
- FIG. 5 is a schematic diagram of a connection manner between a register group and an operation unit in the embodiment of the present application.
- FIG. 6 is a schematic diagram of a connection manner between a register group and an arithmetic unit in still another embodiment of the present application.
- FIG. 7 is a schematic diagram showing the arrangement of data to be convolved in a plurality of register groups in the embodiment of the present application.
- FIG. 8 is a schematic diagram of a process of performing a convolution operation in the embodiment of the present application.
- FIG. 9 is a schematic flowchart of a calculation method in an embodiment of the present application.
- Convolution operations are commonly used in convolutional neural networks.
- a convolutional neural network can be used to process image data.
- the data to be convoluted may be referred to as an input matrix or input data.
- the input data can be image data.
- the convolution operation can correspond to one convolution kernel.
- the convolution kernel is a matrix of K*K size, and K is an integer greater than or equal to 1.
- Each element in the convolution kernel is a weight data.
- the convolution kernel slides on the input matrix according to the step size, and the input matrix is divided into many sub-matrices of the same size as the convolution kernel by the sliding window, and each sub-matrix is multiplied by the convolution kernel, and The dot multiplication result is accumulated to obtain the operation result of the convolution operation.
- the width and length of the convolution core may or may not be equal.
- the width and length of the convolution kernel are equal, that is, the convolution kernel size is K*K.
- FIG. 1 is a schematic structural diagram of a computing device 100 for a convolution operation according to an embodiment of the present application.
- computing device 100 typically includes a convolution processing unit 110, a register file 120, a control unit 130, and a memory 140.
- the convolution processing unit 110 may include a plurality of arithmetic units. Each arithmetic unit is configured to perform a multiplication operation between the data to be convolved and the weight data.
- Convolution processing unit 110 may also include an adder tree unit. The adder tree unit can be used to accumulate the output of the arithmetic unit according to the rules of the convolution calculation to obtain the operation result of the convolution operation.
- the memory 140 is used to store data.
- the register file 120 is used to read the data to be convolved from the memory 140 via the bus and store it in the register file 120.
- Control unit 130 can be used to control convolution processing unit 110 and register file 120 to perform the operations described above.
- a plurality of arithmetic units operate independently of each other. That is, there is no connection relationship between multiple arithmetic units. Therefore, each arithmetic unit needs to read the data to be convolved from the register file 120 separately during operation. There is a large amount of duplicate data in the data to be convolved processed in the multiple convolution operation, but these repeated data still need to be repeatedly read from the register file 120. Therefore, the utilization of data is low.
- Figure 2 is a diagram showing the convolution operation of different period convolution kernels with data to be convolved.
- the convolution kernel is a weight matrix of 2*2
- the data to be convolved is multi-row and multi-column data, which may also be referred to as input data or an input matrix.
- Figure 2 shows the data processed in two adjacent convolution operations. Wherein, the convolution kernel slides on the input matrix according to the step size, and the data covered by the oblique line region of FIG. 2 is repeated data in two operations, which shows the reusability of the convolution operation.
- a computing device that performs a convolution operation generally processes data in parallel using a plurality of arithmetic units.
- Multiple arithmetic units are independent of each other and cannot transmit data to each other. Therefore, multiple arithmetic units need to read the data required for this convolution operation from the register file at each convolution operation. Data that is repeated in two convolution operations is also read repeatedly from the register file. For example, it is assumed that the first arithmetic unit and the second arithmetic unit among the plurality of arithmetic units respectively perform the first convolution operation and the second convolution operation. The first convolution operation and the second convolution operation are adjacent operations. Then the first arithmetic unit reads the data to be convolved 10, 27, 35 and 82 from the register file in the first convolution operation cycle.
- the second arithmetic unit reads the data to be convolved 27, 69, 82, and 75 from the register file in the second convolution operation cycle. Among them, the data 27, 82 is repeatedly read from the register file. When the amount of data processed by the convolution operation is large, a large amount of data is repeatedly read, thereby increasing the pressure for accessing the bandwidth.
- the embodiment of the present application proposes a computing device.
- the computing device is capable of reducing data that is repeatedly read from the register file, reducing access bandwidth. Each period of time can load weight data or data to be convolved for each calculation unit.
- FIG. 3 is a schematic structural diagram of a computing device 300 according to an embodiment of the present invention. As shown in FIG. 3, the computing device 300 includes:
- the plurality of operation units 310 are configured to perform a plurality of convolution operations on the plurality of data, and the plurality of operation units 310 include a first operation unit 311 and a second operation unit 312.
- the first operation unit 311 is configured to perform multiplication on the first weight data and the first data in the plurality of data in the ith convolution operation in the multiple convolution operation, wherein And i is an integer greater than 0; receiving second data transmitted by the second operation unit, wherein the second data is the second operation unit 312 performing a convolution operation in the ith convolution operation Data; in the i+1th convolution operation of the multiple convolution operation, multiplying the first weight data and the second data.
- the receiving the second data sent by the second operation unit 312 may refer to that the first operation unit 311 directly acquires the second data by using a connection with the second operation unit 312. It may also be said that the first operation unit 311 acquires the second data through an indirect connection with the second operation unit 312.
- the first operation unit 311 is connected to the third operation unit
- the third operation unit is connected to the second operation unit 312
- the second operation unit 312 transmits the second data to the third operation unit
- the third operation unit The second data is sent to the first operation unit 311.
- the plurality of operation units 310 are in one-to-one correspondence with the plurality of weight data in the convolution kernel, and are used to respectively perform multiplication operations corresponding to the plurality of weight data in one convolution operation.
- the first weight data is the weight data corresponding to the first operation unit 311
- the second weight data is the weight data corresponding to the second operation unit 312.
- the first operation unit may acquire the data repeated in the two convolution operations from the second operation unit, without reading the repeated data to be convolved from the register file, thereby improving the utilization of the data.
- the rate reduces the amount of data exchanged between the arithmetic unit and the register file, reducing the overhead of access bandwidth between the arithmetic unit and the register file.
- the computing device 300 may further include a plurality of register sets 320 for storing the plurality of data for the multiple convolution operation; the plurality of operation units 310 Also used to retrieve a plurality of data processed by the multiple convolution operations from the plurality of register sets 320.
- the partial operation unit 310 may obtain data that is not repeated in the current convolution operation and the previous convolution operation from the plurality of register sets 320, and the remaining operation unit 310 may obtain the current operation unit 310 from the other operation unit 310. Data that is repeated in the convolution operation with the last convolution operation.
- all of the arithmetic units 310 need to acquire the data to be convolved from the register set 320.
- all the arithmetic units 310 all need to obtain the data to be convolved from the register set 320.
- the plurality of data includes the first data and the second data.
- the first data is stored in a first register set of the plurality of register sets and the second data is stored in a second register set of the plurality of register sets.
- the first register set and the second register set are different register sets. If the data processed in the i-1th convolution operation does not include the first data and the second data, in the ith convolution operation, the first arithmetic unit is from the first register set Acquiring the first data, the second operation unit acquiring the second data from the second register set.
- each register bank includes a plurality of registers, each of which is used to store one data.
- Each register bank can be provided with a read port. That is, each of the plurality of arithmetic units can read one data from one register group per clock cycle.
- the plurality of arithmetic units 310 described above are used to perform a multiplication operation in a convolution operation in each convolution operation.
- the convolution kernel corresponding to the convolution operation includes K*K weight data, and K is an integer greater than one.
- the plurality of operation units 310 may include K*K operation units, and the K*K operation units are respectively used to perform K*K multiplication operations in which the K*K weight data are in one-to-one correspondence.
- the K*K operation units may be divided into K groups of processing units, each group including K operation units, and in one example, the K groups of operation units may correspond to K column weights in K*K weight data.
- the value data, the K operation units in each group of operation units are in turn corresponding to the K weight data in each column weight data.
- the K group operation unit may correspond to the K row weight value data in the K*K weight data, and the K operation units in each group of operation units are sequentially and the K weights in each row of weight data.
- the data corresponds one by one.
- the direction of the row of the convolution kernel along the input matrix is taken as an example for description. Those skilled in the art can understand that the scheme in which the convolution kernel moves along the column direction of the input matrix is similar to the row direction, and details are not described herein again.
- the K group operation unit may be configured to perform K*K multiplication operations corresponding to K*K weight data in the ith convolution operation. After the ith convolution operation is performed, the Kt group to the Kth operation unit in the K group operation unit are used to acquire the i+1th convolution operation from the plurality of register sets 320. Data, where t is the sliding step size corresponding to the convolution operation, and t is an integer greater than or equal to 1.
- the yth group operation unit in the K group operation unit is configured to read the second data from the y+t group operation unit, y is a positive integer, and 1 ⁇ y ⁇ K-t.
- the dth operation unit in the yth group operation unit is configured to read the second data from the dth operation unit in the y+t group operation unit.
- the second data is the data to be convolved that is repeated in the i-th convolution operation and the i+1th convolution operation.
- the first operation unit in FIG. 3 may be any one of the yth operation units.
- the second arithmetic unit in FIG. 3 may be any one of the K-thth to Kth operational units.
- adjacent ones of each of the above K*K arithmetic units are connected to each other.
- This connection method can support data sliding between the arithmetic units.
- a part of the plurality of arithmetic units can acquire non-repeated data from the two convolution operations from the plurality of register sets, and the remaining arithmetic unit can obtain duplicates in two consecutive convolution operations from the adjacent arithmetic unit. The data. This avoids reading duplicate data from multiple register banks to reduce the amount of data that interacts between the arithmetic unit and the register file.
- K*K arithmetic units are not limited to being connected to each other in each row of arithmetic units, and various other connection methods may be employed. For example, adjacent units between the arithmetic units in each column of arithmetic units may be connected to each other. Alternatively, all of the arithmetic units in each row of arithmetic units are connected. Alternatively, other connection methods may be adopted as long as the connection mode enables the first operation unit 311 to acquire the second data from the second operation unit 312.
- FIG. 4 shows a detailed block diagram of a computing device 400 in one embodiment of the present application.
- the computing device 400 includes a convolution calculation unit 31 and a plurality of register sets 320.
- the convolution kernel includes K*K weight data.
- the convolution processing unit 31 includes K*K arithmetic units 310 and an addition tree unit 316.
- the K*K arithmetic units 310 are in one-to-one correspondence with the K*K weight data, and are respectively used to perform a multiplication operation in which the K*K weight data are in one-to-one correspondence.
- the K*K arithmetic units 310 are arranged in K rows and K columns, and adjacent arithmetic units in each row of arithmetic units are connected.
- each of the operation units 310 may include a first register 3101, a second register 3102, and a multiplier 3103.
- the multiplier 3103 is connected to the first register 3101 and the second register 3102.
- the first register 3101 is configured to store weight data corresponding to each of the operation units 310
- the second register 3102 is configured to store data to be convolved corresponding to the operation unit 310 in a single convolution operation.
- the multiplier 3103 is for multiplying the data stored in the first register 3101 and the second register 3102.
- the addition tree unit 316 is for receiving the result of the multiplication operation from the K*K operation units 310, respectively, and performing an accumulation operation on the above result to obtain an operation result of the one convolution operation.
- the computing device 400 described above may further include a memory 330 and a control unit 340.
- the memory 330 can be connected to the plurality of register sets 320 via a bus.
- the memory 330 is used to store data to be convolved and input data to be convolved to a plurality of register banks via a bus.
- the control unit 340 can be used to control the convolution processing unit 31 and the plurality of register sets 320 to perform the above operations.
- the embodiment of the present application does not limit the type of the memory.
- the above memory may be a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM).
- DDR SDRAM Double Data Rate Synchronous Dynamic Random Access Memory
- the manner in which the convolution data is stored in the plurality of register sets 320 is not limited in the embodiment of the present application.
- a plurality of register sets 320 can be partitioned. Each area corresponds to an arithmetic unit 310 that reads only the required data to be convolved from the area without reading data from other areas.
- FIG. 5 illustrates a manner in which a plurality of arithmetic units 310 and a plurality of register sets 320 are connected in the embodiment of the present application.
- the plurality of arithmetic units 310 are in one-to-one correspondence with the plurality of register sets 320, and each of the arithmetic units is connected to a register set.
- Each register set can be used to store data to be convolved corresponding to one arithmetic unit.
- Each arithmetic unit reads in one data from the corresponding register set every clock cycle.
- each operation unit can only read data in the corresponding area, if multiple data to be convolved of the operation unit is repeated, multiple partitions in the plurality of register groups 320 are required.
- This data is repeatedly read and stored from the memory 330.
- This mode of storage occupies the memory space of the plurality of register banks 320 and also increases the bandwidth requirements of the plurality of register banks 320 to read data from the memory. For example, referring to FIG. 1, in a convolution operation, if the first operation unit 311 reads in the data 10, the second operation unit 312 reads in the data 27, and performs a convolution operation after the convolution kernel moves to the right by one column. The first operation unit 311 needs to read in the data 27, and the second operation unit 312 needs to read in the data 69. Then, the data 27 is repeatedly read and stored by the partition corresponding to the first operation unit 311 and the second operation unit 312.
- each arithmetic unit may access data stored in any of the register sets.
- each of the plurality of operation units 310 is configured to acquire data stored in any one of the plurality of register sets 320.
- FIG. 6 illustrates a manner in which a plurality of arithmetic units 310 and a plurality of register sets 320 are connected in still another embodiment of the present application. It should be noted that the connection manner or structure shown in FIG. 6 can be applied to the computing device provided by the present application, such as the computing device of FIG. 1, FIG. 3 or FIG. It can also be applied to any other computing device, which is not limited in this embodiment of the present application.
- a fully connected switched network can be provided between the plurality of arithmetic units 310 and the plurality of register sets 320.
- the fully-connected switching network is a network for implementing a plurality of arithmetic units 310 and a plurality of register sets 320 that are fully connected before.
- the plurality of arithmetic units 310 can be connected to any one of the plurality of register sets 320.
- each of the plurality of arithmetic units is operable to acquire data stored in any one of the plurality of register sets.
- the first operation unit 311 may acquire the first data in the first register group of the plurality of register sets 320, and in the j-th convolution operation, the foregoing An operation unit 311 can acquire third data in a second register group of the plurality of register sets 320.
- i, j is a positive integer, i ⁇ j.
- a full connection switching network is connected between the plurality of operation units 310 and the plurality of register groups 320, so that any one of the plurality of operation units 310 can access any of the register groups,
- the plurality of register sets 320 need only read and store the data to be convolved once, without having to read and store a large amount of data repeated in the multiple convolution operations from the memory 330, thereby saving the storage space of the plurality of register sets 320.
- reducing the pressure of the access bandwidth between the plurality of register sets 320 and the memory 330 improves the degree of parallelism of data access.
- each arithmetic unit 310 can read only one data from one register set 320 in each clock cycle. If a corresponding plurality of data to be convolved in a convolution operation is stored in the same register group 320, there is a data read conflict. Then, the plurality of arithmetic units 310 require a plurality of clock cycles to read all of the required data to be convolved. This storage mode affects the processing speed of the arithmetic unit 310.
- the embodiment of the present application further proposes a manner of storing data to be convolved in a plurality of register sets 320.
- the manner in which a plurality of register sets 320 store data to be convolved will be described below.
- a plurality of data to be convolved in a plurality of convolution operations may be arranged such that during the i-th convolution operation, the plurality of arithmetic units 310 are from the plurality of register sets The data read in 320 is located in a different register set of the plurality of register sets 320.
- the above-mentioned i-th convolution operation may be any one of a plurality of convolution operations.
- the data read by the plurality of arithmetic units 310 from the plurality of register sets 320 may refer to data that needs to be read from the plurality of register sets 320 in the i-th convolution operation.
- the data that needs to be read from the plurality of register sets 320 in the above-described i-th convolution operation may be all data processed in the i-th convolution operation, or may be partial data processed in the i-th convolution operation.
- the repeated data may be acquired from the plurality of operation units 310, and the non-repeated data may be from the plurality of Obtained in register set 320. Therefore, the data that needs to be read from the plurality of register sets 320 in the i-th convolution operation is the above-mentioned non-duplicate data. If there is no duplicate data in the i-th convolution operation and the i-1th convolution operation, the data to be read from the plurality of register sets 320 in the i-th convolution operation is the i-th convolution operation. All data to be processed.
- a flexible data arrangement manner is adopted in a plurality of register groups, so that as much data as possible can be read out in one clock cycle, data read conflicts are avoided, and a non-blocking convolution operation process is realized.
- the convolution operation corresponds to a convolution kernel size of K*K
- the plurality of register sets includes K*K register sets
- the plurality of data includes M rows and N columns of data
- the plurality of The distribution of data in the K*K register sets meets the following conditions:
- the data in the xth row and the sth column of the plurality of data is stored in the bth register, and the data in the xth row and the sth column of the plurality of data is stored in the [b+(a*) K)] mod (K * K) register group, where K is an integer not less than 1, and mod means performing a remainder operation, M ⁇ x ⁇ 1, N ⁇ s ⁇ 1, K * K ⁇ b ⁇ 1 , M, N, x, s, b, a are positive integers.
- each row of the M rows of data may be sequentially arranged in a plurality of register groups.
- the first row of data may be sequentially arranged from the first register set to the K*Kth register set, and then continue to be cyclically arranged from the first register set.
- the plurality of data may be arranged such that: any K consecutive data in each of the M rows of data is stored in a different register set, and/or each column of the N columns of data Any K consecutive data in it is stored in a different register set.
- the foregoing arrangement manner is only an example, and the arrangement manner of the plurality of data may be modified and converted by referring to the above formula.
- the plurality of register sets may not be limited to K*K register sets as long as the data to be convolved in one convolution operation is stored in a different register set.
- the data can still be stored according to the above arrangement.
- FIG. 7 is a specific schematic diagram of a manner in which a plurality of data to be convolved in an embodiment of the present application is arranged in a plurality of register groups.
- the convolution kernel is a weight matrix of 3*3 size.
- the plurality of operation units may include 3*3 operation units, and the plurality of register groups may include 3*3 register groups.
- a plurality of arithmetic units and a plurality of register sets are connected by a fully connected switching network.
- the plurality of register sets described above may constitute a register array. Each row represents a register bank, and each register bank includes a number of registers.
- the row number of the register array can be represented by column numbers r1, r2, ... rn.
- the row number of the register array can be represented by b1, b2, ... b9.
- Also shown in Figure 7 is a plurality of data to be convolved data required for a multiple convolution operation.
- the plurality of data described above includes a plurality of rows and columns of data. For ease of representation, the above data is represented by 1, 2, 3, ..., 256, respectively.
- the above plurality of data can be arranged in a plurality of register groups in accordance with the manner described above.
- the first line of data 1, 2, ..., 16 may be sequentially stored in the first register group b1 to the ninth register b9 in a loop.
- the second row of data is loaded from the [b+(a*K)] mod(K*K) group register.
- the second row of data may be cyclically stored in a plurality of register banks starting from the fourth register bank.
- the third row of data can be cyclically stored in multiple register banks starting from the seventh register bank.
- FIG. 8 is a specific example diagram of a convolution operation in still another embodiment of the present application. It is assumed that the convolution kernel in Fig. 8 is a 3*3 convolution kernel, which employs the computing device shown in Fig. 3 or Fig. 4 and the data arrangement in Fig. 7.
- the process of the convolution operation is as follows:
- the weight data corresponding to the 3*3 operation units 310 is read from the plurality of register sets 320, and the weight data is stored in the first register 3101 in each of the operation units 310. Since the weight data is arranged in an order, when it is loaded into the arithmetic unit 310, collision-free reading can be realized.
- the first three data of the first three rows in the data to be convolved are read to the plurality of operation units 310, and a total of 3*3 data, due to the row of the data to be convolved.
- the above 3*3 data are read from the different register sets 320 in the above 3*3 register sets 320, so that the data required for the first convolution operation can be read in one clock cycle.
- the above-described 3*3 arithmetic units 310 respectively perform multiplication operations corresponding to 3*3 weight data one by one, and output the result.
- Each operation unit 310 sends the output result of the 3*3 operation units 310 to the addition tree unit 316 (accumulation unit), obtains the operation result of the first convolution operation, and can write the operation result back to the register file.
- the register file may include the plurality of register sets 320 described above, and may further include other register sets. When the operation result is written back to the register file, the storage location of the operation result does not conflict with the position of the previously stored data to be convolved.
- the third column operation unit 310 reads the fourth data of the first three rows of the plurality of data from the plurality of register sets 320.
- the third column operation unit 310 passes the processed data in the first convolution operation to the second column operation unit 310, and the second column operation unit 310 processes the data in the first convolution operation. It is passed to the first column operation unit 310. Then, each arithmetic unit 310 performs a multiplication operation in the second convolution operation.
- each operation unit 310 sends the output result of the 3*3 operation units 310 to the addition tree unit 316 (accumulation unit), obtains the operation result of the second convolution operation, and can write the operation result back to the plurality of register groups. 320.
- the subsequent convolution process can repeat the above operation.
- the method embodiments of the present application are described below.
- the method embodiments correspond to the device embodiments. Therefore, the parts that are not described in detail may be referred to the foregoing device embodiments.
- FIG. 9 is a schematic flowchart of a calculation method of an embodiment of the present application.
- the computing device to which the computing method is applied includes a plurality of computing units for performing a plurality of convolution operations on a plurality of data, the plurality of computing units including a first computing unit and a second computing unit;
- the method includes:
- the first operation unit multiplies the first weight data and the first data of the plurality of data, where, i Is an integer greater than 0;
- the first operation unit receives second data sent by the second operation unit, where the second data is performed by the second operation unit in the ith convolution operation. data;
- the first operation unit multiplies the first weight data and the second data.
- the computing device of FIG. 9 further includes a plurality of register sets for storing the plurality of data; the computing method of FIG. 9 further comprising: controlling the plurality of computing units from the plurality of computing devices The plurality of data required in the multiple convolution operation are acquired in a register set.
- each of the plurality of operation units is configured to acquire data stored in any one of the plurality of register sets.
- data read by the plurality of operation units from the plurality of register sets is different in the plurality of register sets In the register group.
- the convolution operation corresponds to a convolution kernel size of K*K
- the plurality of register sets includes K*K register sets
- the plurality of data includes M rows N.
- Column data, the distribution of the plurality of data in the K*K register sets is in accordance with the condition that data in the xth row and the sth column of the plurality of data is stored in the rth group register,
- the data in the xth row and the sth column of the plurality of data is stored in the [r+(a*K)] mod(K*K) group register, where K is an integer not less than 1, and mod indicates that For the remainder operation, M ⁇ x ⁇ 1, N ⁇ s ⁇ 1, K*K ⁇ r ⁇ 1, M,N,x,s,r,a are positive integers.
- a computing device for supporting sliding data to be convolved between computing units which can support multiple computing units to read data to be convolved from adjacent computing units, thereby avoiding multiple
- a large amount of duplicate data is read in the register group, which reduces the amount of data exchanged between the arithmetic unit and the plurality of register groups, reduces data access conflicts, and improves the computing performance of the architecture.
- a plurality of arithmetic units and a plurality of register sets may be connected by a fully connected switching network, so that any one of the plurality of arithmetic units can access any of the register sets, so the plurality of register sets need only be read in.
- the disclosed systems, devices, and methods may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
- the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product.
- the technical solution of the present application which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
- the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
- the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Complex Calculations (AREA)
Abstract
本申请提供了一种计算装置和计算方法,能够提高数据利用率。该计算装置包括:多个运算单元,用于对多个数据执行多次卷积操作,多个运算单元包括第一运算单元和第二运算单元;第一运算单元用于:在多次卷积操作中的第i次卷积操作中,对第一权值数据以及多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收第二运算单元发送的第二数据,其中,第二数据是第二运算单元在第i次卷积操作中执行卷积操作的数据;在多次卷积操作中第i+1次卷积操作中,对第一权值数据以及第二数据进行乘法运算。
Description
本申请涉及数据处理领域,尤其涉及一种计算装置和计算方法。
卷积操作是卷积神经网络中的最常用的计算操作。卷积操作主要应用于卷积层中。在卷积操作中,卷积核中的每一个权值数据与其对应的输入数据相乘,然后将点乘结果相加,得到一次卷积操作的输出结果。之后根据卷积层的步长设定,滑动卷积核,重复上述卷积操作。卷积神经网络处理的数据通常数量庞大,高的数据吞吐量需要高的访问带宽。例如,用于卷积运算的计算装置通常包括多个并行处理数据的运算单元和寄存器堆,寄存器堆用于存储待进行卷积运算的数据。在每次卷积操作中,多个运算单元都需要从寄存器堆读取卷积操作所需的数据,由于卷积操作处理的数据量通常很庞大,因而多个运算单元在卷积操作的过程中需要从寄存器堆读取大量的数据。
发明内容
本申请提供一种计算装置和计算方法,能够提高数据利用率。
第一方面,提供了一种计算装置,包括:多个运算单元,用于对多个数据执行多次卷积操作,所述多个运算单元包括第一运算单元和第二运算单元;所述第一运算单元用于:在所述多次卷积操作中的第i次卷积操作中,对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作中的第i+1次卷积操作中,对所述第一权值数据以及所述第二数据进行乘法运算。
在本申请实施例中,第一运算单元可以从第二运算单元中获取两次卷积操作中重复的数据,而无需从寄存器堆中读取重复的待卷积数据,从而提高了数据的利用率,减少了运算单元与寄存器堆之间交互的数据量,减少了运算单元与寄存器堆之间的访问带宽的开销。
在一种可能的实现方式中,还包括:多个寄存器组,用于存储所述多个数据;所述多个运算单元还用于从所述多个寄存器组中获取所述多次卷积操作中所需的所述多个数据,其中,所述多个运算单元中的每个运算单元能够获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
在本申请实施例中,多个运算单元中的任一运算单元能够访问任一寄存器组,因此上述多个寄存器组只需读入和存储一次待卷积数据,无需从存储器多次读取以及存储多次卷积操作中重复的大量数据,因而可以节约多个寄存器组的存储空间,并且减小多个寄存器组和存储器之间的访问带宽的压力,提高了数据访问的并行度。
在一种可能的实现方式中,所述第一运算单元被配置为:在所述第i次卷积操作中,从所述多个寄存器组中的第一寄存器组中获取所述第一数据;所述第二运算单元被配置为:在所述第i次卷积操作中,从所述多个寄存器组中的第二寄存器组中获取所述第二数 据,所述第一寄存器组和所述第二寄存器组为不同的寄存器组。
在一种可能的实现方式中,在所述第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中。
在本申请实施例中,在多个寄存器组中采用了灵活数据排布方式,可以在一个时钟周期内读出尽可能多的数据,避免了数据读取冲突,实现无阻塞卷积运算过程。
在一种可能的实现方式中,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第b个寄存器组中,所述多个数据中的第x+a行第s列中的数据存储于第[b+(a*K)]mod(K*K)个寄存器组中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥b≥1,M,N,x,s,b,a为正整数。
第二方面,提供了一种计算方法,包括:应用所述计算方法的计算装置包括多个运算单元,所述多个运算单元用于对多个数据执行多次卷积操作,所述多个运算单元包括第一运算单元和第二运算单元;所述方法包括:在所述多次卷积操作中的第i次卷积操作中,所述第一运算单元对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;所述第一运算单元接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作中的第i+1次卷积操作中,所述第一运算单元对所述第一权值数据以及所述第二数据进行乘法运算。
在本申请实施例中,第一运算单元可以从第二运算单元中获取两次卷积操作中重复的数据,而无需从寄存器堆中读取重复的待卷积数据,从而提高了数据的利用率,减少了运算单元与寄存器堆之间交互的数据量,减少了运算单元与寄存器堆之间的访问带宽的开销。
在一种可能的实现方式中,所述计算装置还包括:多个寄存器组,用于存储所述多个数据;其中,所述多个运算单元中的每个运算单元能够获取用于获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
在本申请实施例中,多个运算单元中的任一运算单元能够访问任一寄存器组,因此上述多个寄存器组只需读入和存储一次待卷积数据,无需从存储器多次读取以及存储多次卷积操作中重复的大量数据,因而可以节约多个寄存器组的存储空间,并且减小多个寄存器组和存储器之间的访问带宽的压力,提高了数据访问的并行度。
在一种可能的实现方式中,还包括:在所述第i次卷积操作中,所述第一运算单元从所述多个寄存器组中的第一寄存器组中获取所述第一数据;在所述第i次卷积操作中,所述第二运算单元从所述多个寄存器组中的第二寄存器组中获取所述第二数据,所述第一寄存器组和所述第二寄存器组为不同的寄存器组。
在一种可能的实现方式中,在所述第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中。
在本申请实施例中,在多个寄存器组中采用了灵活数据排布方式,可以在一个时钟周期内读出尽可能多的数据,避免了数据读取冲突,实现无阻塞卷积运算过程。
在一种可能的实现方式中,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器 组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第r组寄存器中,所述多个数据中的第x+a行第s列中的数据存储于第[r+(a*K)]mod(K*K)组寄存器中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥r≥1,M,N,x,s,r,a为正整数。
第三方面,提供了一种计算装置,包括多个运算单元和多个寄存器组,其中,每个运算单元分别连接所述多个寄存器组;所述多个寄存器组,用于存储待执行多次卷积操作的多个数据;所述多个运算单元,用于从所述多个寄存器组中获取所述多个数据,并对所述多个数据执行多次卷积操作,其中,所述多个运算单元中的每个运算单元用于获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
在本申请实施例中,多个运算单元中的任一运算单元能够访问任一寄存器组,因此上述多个寄存器组只需读入和存储一次待卷积数据,无需从存储器多次读取以及存储多次卷积操作中重复的大量数据,因而可以节约多个寄存器组的存储空间,并且减小多个寄存器组和存储器之间的访问带宽的压力,提高了数据访问的并行度。
在一种可能的实现方式中,在所述多次卷积操作中的第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中,其中,i为大于0的整数。
在本申请实施例中,在多个寄存器组中采用了灵活数据排布方式,可以在一个时钟周期内读出尽可能多的数据,避免了数据读取冲突,实现无阻塞卷积运算过程。
在一种可能的实现方式中,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第b个寄存器组中,所述多个数据中的第x+a行第s列中的数据存储于第[b+(a*K)]mod(K*K)个寄存器组中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥b≥1,M,N,x,s,b,a为正整数。
在一种可能的实现方式中,所述多个运算单元包括第一运算单元和第二运算单元,所述第一运算单元用于:在所述多次卷积操作中的第i次卷积操作中,对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作中的第i+1次卷积操作中,对所述第一权值数据以及所述第二数据进行乘法运算。
在本申请实施例中,第一运算单元可以从第二运算单元中获取两次卷积操作中重复的数据,而无需从寄存器堆中读取重复的待卷积数据,从而提高了数据的利用率,减少了运算单元与寄存器堆之间交互的数据量,减少了运算单元与寄存器堆之间的访问带宽的开销。
第四方面,提供了一种芯片,所述芯片上设置有如第一方面或第一方面中任一种可能的实现方式中所述的计算装置,或者,所述芯片上设置有如第三方面或第三方面中任一种可能的实现方式中所述的计算装置。
第五方面,提供了一种计算机存储介质,所述存储用于计算装置执行的程序代码,所 述程序代码包括用于执行第二方面的方法的指令。
图1是本申请实施例提供的计算装置的结构示意图。
图2是不同周期卷积核与待卷积数据进行卷积操作的示意图。
图3是本申请实施例的计算装置的结构示意图。
图4是本申请又一实施例的计算装置的结构示意图。
图5是本申请实施例中的寄存器组与运算单元的连接方式示意图。
图6是本申请又一实施例中的寄存器组与运算单元的连接方式示意图。
图7是本申请实施例中的待卷积数据在多个寄存器组中的排布方式示意图。
图8是本申请实施例中的进行卷积操作的过程示意图。
图9是本申请实施例中的计算方法的流程示意图。
下面将结合附图,对本申请中的技术方案进行描述。
为了便于理解,先对卷积操作的概念以及相关技术进行详细介绍。
卷积操作通常应用于卷积神经网络中。卷积神经网络可用于处理图像数据。待进行卷积计算的数据可以称为输入矩阵或输入数据。该输入数据可以是图像数据。其中,卷积操作可对应一个卷积核。卷积核是一个K*K大小的矩阵,K为大于或等于1的整数。卷积核中的每个元素为一个权值数据。在卷积过程中,卷积核根据步长在输入矩阵上滑动,输入矩阵会被滑动窗口划分为许多与卷积核大小相同的子矩阵,每个子矩阵与卷积核进行点乘,并将点乘结果进行累加,以得到卷积操作的运算结果。需要说明的是,卷积核的宽度和长度可以相等,也可以不相等。本申请实施例中,以卷积核的宽度和长度相等进行说明,即卷积核大小为K*K。本领域技术人员能够理解,本申请实施例中的计算装置和计算方法可应用于卷积核的宽度和长度不相等的情形。
图1是本申请实施例提供的用于卷积操作的计算装置100的结构示意图。如图1所示,计算装置100通常包括卷积处理单元110,寄存器堆120,控制单元130以及存储器140。其中,卷积处理单元110可包括多个运算单元。每个运算单元用于执行待卷积数据和权值数据之间的乘法运算。卷积处理单元110还可包括加法器树单元。该加法器树单元可用于根据卷积计算的规则,将运算单元的输出结果进行累加,以得到卷积操作的运算结果。存储器140用于存储数据。而寄存器堆120用于通过总线从存储器140中读取待卷积的数据,并存储在寄存器堆120中。控制单元130可以用于控制卷积处理单元110和寄存器堆120实现上述操作。在相关技术中,多个运算单元之间是相互独立运行的。即多个运算单元之间没有连接关系。所以各个运算单元在工作时需要分别从寄存器堆120中读取待卷积数据。在多次卷积操作中所处理的待卷积数据中存在大量重复的数据,但这些重复的数据依然需要从寄存器堆120中重复读取。因此对数据的利用率较低。
为了便于理解,下面结合图2对卷积操作的过程进行举例说明。
图2示出了不同周期卷积核与待卷积数据进行卷积操作的示意图。如图2所示,卷积核为2*2的权值矩阵,待卷积数据为多行多列数据,也可以称为输入数据或输入矩阵。图 2示出了在两次相邻的卷积操作中所处理的数据。其中,卷积核根据步长在输入矩阵上滑动,图2斜线区域覆盖的数据是两次操作中的重复的数据,这显示了卷积运算的可复用性。但是在相关技术中,进行卷积操作的计算装置通常采用多个运算单元并行处理数据。多个运算单元之间是相互独立的,互相之间不能进行数据传输。因此,多个运算单元在每次卷积操作时都需要从寄存器堆中读取本次卷积操作需要的数据。两次卷积操作中重复的数据也会重复地从寄存器堆中读取。例如,假设多个运算单元中的第一运算单元和第二运算单元分别执行第一卷积操作和第二卷积操作。第一卷积操作和第二卷积操作为相邻的操作。则第一运算单元在第一卷积操作周期中从寄存器堆读取待卷积数据10、27、35和82。第二运算单元在第二卷积操作周期中从寄存器堆中读取待卷积数据27、69、82和75。其中,数据27,82被重复地从寄存器堆中读取。在卷积操作处理的数据量较大时,大量数据被重复读取,因而增加了访问带宽的压力。
本申请实施例提出了一种计算装置。该计算装置能够减少从寄存器堆中重复读取的数据,减少对访问带宽。每个周期可以同时为每个计算单元加载权值数据或待卷积数据。
图3为本发明实施例提供的一种计算装置300的结构示意图。如图3所示,该计算装置300包括:
多个运算单元310,用于对多个数据执行多次卷积操作,所述多个运算单元310包括第一运算单元311和第二运算单元312。所述第一运算单元311用于:在所述多次卷积操作中的第i次卷积操作中,对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元312在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作中的第i+1次卷积操作中,对所述第一权值数据以及所述第二数据进行乘法运算。
可选地,上述接收第二运算单元312发送的第二数据,可以指第一运算单元311通过与第二运算单元312之间的连接直接获取第二数据。也可以指第一运算单元311通过与第二运算单元312之间的间接连接获取第二数据。例如,第一运算单元311与第三运算单元相连,第三运算单元与第二运算单元312相连,第二运算单元312向第三运算单元发送所述第二数据,第三运算单元再将所述第二数据发送给第一运算单元311。
上述多个运算单元310与卷积核中的多个权值数据一一对应,并用于在一次卷积操作中,分别执行多个权值数据对应的乘法操作。例如,上述第一权值数据为第一运算单元311对应的权值数据,上述第二权值数据为第二运算单元312对应的权值数据。
在本申请实施例中,第一运算单元可以从第二运算单元中获取两次卷积操作中重复的数据,而无需从寄存器堆中读取重复的待卷积数据,从而提高了数据的利用率,减少了运算单元与寄存器堆之间交互的数据量,减少了运算单元与寄存器堆之间的访问带宽的开销。
可选地,上述计算装置300还可以包括多个寄存器组320,所述多个寄存器组320用于存储用于所述多次卷积操作的所述多个数据;所述多个运算单元310还用于从所述多个寄存器组320中获取所述多次卷积操作所处理的多个数据。例如,上述部分运算单元310可以从所述多个寄存器组320中获取本次卷积操作中与上次卷积操作中不重复的数据,剩余运算单元310可以从其他运算单元310中获取本次卷积操作中与上次卷积操作中重复的数据。可选地,在一些卷积操作中,由于该次卷积操作与上一次卷积操作之间没有重复的 数据,则所有运算单元310需要从寄存器组320中获取待卷积的数据。例如,在第一次卷积操作中,或者当卷积核按行方向移动,并移动到新的一行时,或者当卷积核按列方向移动,并移动到新的一列时,所有运算单元310均需要从寄存器组320中获取待卷积的数据。
上述多个数据包括上述第一数据和上述第二数据。在一个示例中,假设第一数据存储于多个寄存器组中的第一寄存器组,第二数据存储于所述多个寄存器组中的第二寄存器组。第一寄存器组和第二寄存器组为不同的寄存器组。若第i-1次卷积操作中处理的数据不包括上述第一数据和上述第二数据,则在所述第i次卷积操作中,所述第一运算单元从所述第一寄存器组中获取所述第一数据,所述第二运算单元从所述第二寄存器组中获取所述第二数据。
可选地,上述多个寄存器组320也可以称为寄存器堆。每个寄存器组包括多个寄存器,每个寄存器用于存储一个数据。每个寄存器组可设有一个读端口。即上述多个运算单元中的每个运算单元在每个时钟周期内能够从一个寄存器组中读取一个数据。
在一些示例中,上述多个运算单元310用于在每次卷积操作中,执行卷积操作中的乘法运算。例如,假设卷积操作对应的卷积核包括K*K个权值数据,K为大于1的整数。则多个运算单元310可以包括K*K个运算单元,所述K*K个运算单元分别用于执行K*K个权值数据一一对应的K*K个乘法操作。
所述K*K个运算单元可以划分为K组处理单元,每组包括K个运算单元,在一个示例中,所述K组运算单元可以对应于K*K个权值数据中的K列权值数据,每组运算单元中的K个运算单元依次与每列权值数据中的K个权值数据一一对应。或者,则所述K组运算单元可以对应于K*K个权值数据中的K行权值数据,每组运算单元中的K个运算单元依次与每行权值数据中的K个权值数据一一对应。在本申请实施例中,以卷积核沿输入矩阵的行方向为例进行说明。本领域技术人员能够理解,卷积核沿输入矩阵的列方向移动的方案与行方向类似,此处不再赘述。
所述K组运算单元可用于:在上述第i次卷积操作中,分别执行K*K个权值数据对应的K*K个乘法操作。在执行完所述第i次卷积操作之后,所述K组运算单元中的第K-t组至第K组运算单元用于从多个寄存器组320中获取第i+1次卷积操作所需的数据,其中,t为所述卷积操作对应的滑动步长,t为大于等于1的整数。所述K组运算单元中的第y组运算单元用于从第y+t组运算单元中读取所述第二数据,y为正整数,且1≤y≤K-t。其中,第y组运算单元中的第d个运算单元用于从第y+t组运算单元中的第d个运算单元读取所述第二数据。上述第二数据为第i次卷积操作与第i+1次卷积操作中重复的待卷积数据。
可选地,图3中的第一运算单元可以是第y组运算单元中的任一运算单元。图3中的第二运算单元可以是第K-t至第K组运算单元中的任一运算单元。
在一些示例中,上述K*K个运算单元中的每行运算单元中的相邻运算单元相互连接。该连接方式能够支持数据在运算单元之间滑动。从而多个运算单元中的部分运算单元可以从多个寄存器组中获取两次卷积操作中非重复的数据,而剩余运算单元可以从相邻的运算单元获取两次连续的卷积操作中重复的数据。从而避免从多个寄存器组中读取重复数据,以减少运算单元与寄存器堆之间交互的数据量。
需要说明的是,上述K*K个运算单元之间并不限于每行运算单元之间相互连接,也 可以采用其他多种连接方式。例如,可以是每列运算单元中的运算单元之间的相邻单元相互连接。或者,是每行运算单元中的运算单元之间全部相连。或者也可以采取其他连接方式,只要该连接方式能够使得第一运算单元311能够从第二运算单元312获取第二数据即可。
作为一个示例,图4示出了本申请一个实施例中的计算装置400的具体结构图。如图4所示,该计算装置400包括卷积计算单元31和多个寄存器组320。假设卷积核包括K*K个权值数据。所述卷积处理单元31包括K*K个运算单元310以及加法树单元316。K*K个运算单元310与K*K个权值数据一一对应,分别用于执行K*K个权值数据一一对应的乘法操作。K*K个运算单元310排列为K行K列,每行运算单元中的相邻运算单元相连。具体地,所述每个运算单元310可包括第一寄存器3101、第二寄存器3102以及一个乘法器3103。所述乘法器3103与第一寄存器3101、第二寄存器3102相连。第一寄存器3101用于存储每个运算单元310对应的权值数据,第二寄存器3102可用于存储该运算单元310在一次卷积操作中对应的待卷积数据。乘法器3103用于对第一寄存器3101和第二寄存器3102存储的数据进行乘法操作。加法树单元316用于分别从K*K个运算单元310接收乘法操作的结果,并对上述结果进行累加操作,以得到一次卷积操作的运算结果。
可选地,上述计算装置400还可以包括存储器330和控制单元340。其中,存储器330可以通过总线与多个寄存器组320相连。存储器330用于存储待卷积的数据,并通过总线向多个寄存器组输入待卷积的数据。控制单元340可以用于控制卷积处理单元31和多个寄存器组320实现上述操作。
本申请实施例对存储器的类型不作限定。例如,上述存储器可以是双倍速率同步动态随机存储器(Double Data Rate Synchronous Dynamic Random Access Memory,DDR SDRAM)。
可选地,本申请实施例对待卷积数据在多个寄存器组320中的存储方式不作限定。例如,可以对多个寄存器组320进行分区划分。每个区域对应于一个运算单元310,该运算单元310仅从该区域读取所需的待卷积数据,而不从其他区域读取数据。
作为一个示例,图5示出了本申请实施例中的多个运算单元310与多个寄存器组320的连接方式。多个运算单元310与多个寄存器组320一一对应,每个运算单元与一个寄存器组相连。每个寄存器组可以用于存储一个运算单元对应的待卷积数据。每个运算单元在每个时钟周期内从对应的寄存器组读入一个数据。
但是,在分区的存储方式中,由于每个运算单元只能读取对应区域中的数据,因此,若多个运算单元的待卷积数据重复时,多个寄存器组320中的多个分区需要从存储器330中重复地读取和存储该数据。这种存储方式占用了多个寄存器组320的存储空间,也增加了多个寄存器组320从存储器读入数据的带宽要求。例如,继续参见图1,在一次卷积操作中,若第一运算单元311读入数据10,第二运算单元312读入数据27,在卷积核向右移动一列后进行的卷积操作中,第一运算单元311需读入数据27,第二运算单元312需读入数据69。则数据27会被第一运算单元311和第二运算单元312对应的分区重复读入和存储。
可选地,也可以不对多个寄存器组320进行分区,每个运算单元可以访问任何寄存器组中存储的数据。在不分区的方式下,所述多个运算单元310中的每个运算单元310用于 获取所述多个寄存器组320中的任意一个寄存器组中存储的数据。
作为一个示例,图6示出了本申请又一实施例中的多个运算单元310与多个寄存器组320的连接方式。需要说明的是,图6所示的连接方式或结构可以应用本申请提供的计算装置中,例如图1、图3或图4的计算装置。也可以应用其他任何计算装置中,本申请实施例对此不作限定。
如图6所示,可以在多个运算单元310和多个寄存器组320之间设置全连接交换网络。全连接交换网络是用于实现多个运算单元310和多个寄存器组320之前全相连的网络。通过该全连接交换网络,多个运算单元310可以与多个寄存器组320中的任意一个寄存器相连。换句话说,所述多个运算单元中的每个运算单元可用于获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
例如,在第i次卷积操作中,上述第一运算单元311可以在所述多个寄存器组320中的第一寄存器组中获取上述第一数据,在第j次卷积操作中,上述第一运算单元311可以在所述多个寄存器组320中的第二寄存器组中获取第三数据。其中,i,j为正整数,i≠j。
在本申请实施例中,在多个运算单元310和多个寄存器组320之间采用了全连接交换网络相连,使得多个运算单元310中的任一运算单元310能够访问任一寄存器组,因此上述多个寄存器组320只需读入和存储一次待卷积数据,无需从存储器330多次读取以及存储多次卷积操作中重复的大量数据,因而可以节约多个寄存器组320的存储空间,并且减小多个寄存器组320和存储器330之间的访问带宽的压力,提高了数据访问的并行度。
但是在不分区的存储方式中,由于每个寄存器组320只对应一个读端口,因此每个运算单元310在每个时钟周期内仅能从一个寄存器组320读入一个数据。若在一次卷积操作中对应的多个待卷积数据存储于同一个寄存器组320中时,存在数据读取冲突。则多个运算单元310需要多个时钟周期才能把所需的待卷积数据全部读出。这种存储方式会影响运算单元310的处理速度。
为了解决上述问题,本申请实施例还进一步提出在多个寄存器组320中存储待卷积数据的方式。下面将继续描述多个寄存器组320存储待卷积数据的方式。
在一些实施例中,可以对多次卷积操作中待卷积的多个数据进行排布,使得在第i次卷积操作过程中,所述多个运算单元310从所述多个寄存器组320中读取的数据位于所述多个寄存器组320的不同寄存器组中。
其中,上述第i次卷积操作可以是多次卷积操作中的任意一次卷积操作。上述多个运算单元310从所述多个寄存器组320中读取的数据可指在第i次卷积操作中需要从所述多个寄存器组320中读取的数据。上述第i次卷积操作中需要从多个寄存器组320中读取的数据可以是第i次卷积操作中的处理的全部数据,也可以是第i次卷积操作处理的部分数据。例如,若第i次卷积操作中与第i-1次卷积操作中存在部分重复的数据,则重复的数据可从多个运算单元310内部获取,不重复的数据可从所述多个寄存器组320中获取。因此第i次卷积操作中需要从多个寄存器组320中读取的数据为上述不重复的数据。若第i次卷积操作与第i-1次卷积操作中不存在重复的数据,则第i次卷积操作中需要从多个寄存器组320中读取的数据为第i次卷积操作中待处理的全部数据。
在本申请实施例中,在多个寄存器组中采用了灵活数据排布方式,可以在一个时钟周期内读出尽可能多的数据,避免了数据读取冲突,实现无阻塞卷积运算过程。
对上述待卷积的多个数据进行排布存在多种具体方式。在一个示例中,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:
所述多个数据中的第x行第s列中的数据存储于第b个寄存器中,所述多个数据中的第x+a行第s列中的数据存储于第[b+(a*K)]mod(K*K)个寄存器组中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥b≥1,M,N,x,s,b,a为正整数。
可选地,在符合上述条件的基础上,上述M行数据中的每行数据可以在多个寄存器组中依次循环排布。例如,第一行数据可以从第一寄存器组依次排布至第K*K个寄存器组,然后继续从第一寄存器组开始进行循环排布。
可选地,可以对上述多个数据进行排布使得:上述M行数据中的每行数据中的任意K个连续数据存储在不同寄存器组中,和/或上述N列数据中的每列数据中的任意K个连续数据存储在不同的寄存器组中。
其中,在一些实施例中,上述排布方式仅作为一个示例,也可以参照上述公式对多个数据的排布方式进行修改和转换。例如,上述多个寄存器组也可以不限于K*K个寄存器组,只要一次卷积操作中待卷积的数据存储于不同的寄存器组即可。或者,上述多个数据中的行和列可以进行反转之后,仍然可以根据上述排布方式存储数据。
图7是本申请实施例中的多个待卷积数据在多个寄存器组中的排布方式的具体示意图。为了便于理解,下面结合图7对多个待卷积数据的具体排布方式进行说明。如图7所示,卷积核为3*3大小的权值矩阵。上述多个运算单元可以包括3*3个运算单元,上述多个寄存器组可以包括3*3个寄存器组。多个运算单元与多个寄存器组之间通过全连接交换网络相连。上述多个寄存器组可以构成寄存器阵列。每行表示一个寄存器组,每个寄存器组包括若干寄存器。其中,寄存器阵列的行号可以用列号用r1,r2,…rn表示。寄存器阵列的行号可以用b1,b2,…b9表示。图7中还示出了多次卷积操作所需的多个待卷积数据。上述多个数据包括多行多列数据。为了便于表示,上述数据分别用1,2,3,…,256表示。
上述多个数据可以根据前文中的方式在多个寄存器组中排布。例如,第一行数据1,2,…,16可以依次循环地存入第一寄存器组b1至第九寄存器b9之中。若第一行数据中的第一个数据存储入第b个寄存器,则第二行数据从第[b+(a*K)]mod(K*K)组寄存器开始加载数据。例如,第一行数据中的第一个数据1位于第一个寄存器组中,即b=1,根据上述计算方式,第二行数据中的第一个数据17存储于第[1+(1*3)]mod 9=4个寄存器组中。而第三行数据中的第一个数据33应该存储于第[1+(2*3)]mod 9=7个寄存器组中。作为一种可选地方式,第二行数据可以从第4个寄存器组开始依次循环地存储入多个寄存器组中。第三行数据可以从第7个寄存器组开始循环地存储入多个寄存器组中。采用上述排布方式的情况下,多个运算单元可以在一个时钟周期内读取一次卷积操作所需的所有数据。另外,如图7所示,假设3*3个权值数据分别为q1-q9。则上述权值数据也可以分别存入多个寄存器组中的不同寄存器组中。采用上述排布方式的情况下,在多次卷积操作开始之前,多个运算单元可以在一个时钟周期内读取所有的权值数据,然后再读取卷积操作中的待卷积数据。
为了便于理解,下面结合图8,继续详细介绍本申请实施例中的计算装置进行卷积操作的处理过程。图8是本申请又一实施例中的卷积操作的具体示例图。假设图8中的卷积核为3*3的卷积核,其采用了图3或图4所示的计算装置以及图7中的数据排布方式。卷积操作的过程如下:
S801、从存储器330中加载卷积核对应的权值数据至多个寄存器组320中,3*3个权值数据各存储入不同的寄存器组320中。
S802、加载多次卷积操作对应的多个数据到多个寄存器组320中,排布规则如图7所示。
S803、从多个寄存器组320中读取至3*3个运算单元310对应的权值数据,并将权值数据存储入每个运算单元310中的第一寄存器3101中。由于权值数据有序排列,因此将其载入运算单元310时,可实现无冲突读取。
S804、在第一次卷积操作中,将待卷积数据中的前3行的前3个数据读取至多个运算单元310,共3*3个数据,由于上述的待卷积数据的排布规则,上述3*3个数据从上述3*3个寄存器组320中的不同寄存器组320中读出,因此可以在一个时钟周期中读出第一次卷积操作所需的数据。上述3*3个运算单元310分别进行与3*3个权值数据一一对应的乘法操作,并输出结果。
S805、各运算单元310将3*3个运算单元310的输出结果送入加法树单元316(累加单元),获取第一次卷积操作的运算结果,并可以将运算结果写回寄存器堆中。其中,所述寄存器堆可包括上述多个寄存器组320,还可以包括其他寄存器组。所述运算结果在写回寄存器堆时,所述运算结果的存储位置不与之前存储的待卷积数据的位置相冲突。
S806、在进行第二次卷积运算之前,第三列运算单元310从多个寄存器组320中读取多个数据的前3行的第4个数据;
S807、第三列运算单元310将第一次卷积操作中的处理的数据传递至第二列运算单元310中,第二列运算单元310将其在第一次卷积操作中的处理的数据传递至第一列运算单元310中。然后,各运算单元310进行第二次卷积操作中的乘法运算。
S808、各运算单元310将3*3个运算单元310的输出结果送入加法树单元316(累加单元),获取第二次卷积操作的运算结果,并可以将运算结果写回多个寄存器组320中。之后的卷积过程可重复上述操作。
下面对本申请的方法实施例进行描述,方法实施例与装置实施例对应,因此未详细描述的部分可以参见前面各装置实施例。
图9是本申请实施例的计算方法的示意性流程图。应用所述计算方法的计算装置包括多个运算单元,所述多个运算单元用于对多个数据执行多次卷积操作,所述多个运算单元包括第一运算单元和第二运算单元;
所述方法包括:
S901、在所述多次卷积操作中的第i次卷积操作中,所述第一运算单元对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;
S902、所述第一运算单元接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;
S903、在所述多次卷积操作中的第i+1次卷积操作中,所述第一运算单元对所述第一 权值数据以及所述第二数据进行乘法运算。
可选地,图9中的计算装置还包括多个寄存器组,用于存储所述多个数据;图9中的计算方法还包括:控制所述多个运算单元从所述计算装置的多个寄存器组中获取所述多次卷积操作中所需的所述多个数据。
可选地,在图9的方法中,所述多个运算单元中的每个运算单元用于获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
可选地,在图9的方法中,在所述第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中。
可选地,在图9的方法中,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第r组寄存器中,所述多个数据中的第x+a行第s列中的数据存储于第[r+(a*K)]mod(K*K)组寄存器中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥r≥1,M,N,x,s,r,a为正整数。
在本申请实施例中,提供了一种支持运算单元间滑动待卷积数据的计算装置,能支持多个运算单元从邻近的运算单元中读取待卷积数据,以此来避免从多个寄存器组中读取大量重复数据,减少了运算单元与多个寄存器组之间交互的数据量,减少数据访问冲突,提高了架构的运算性能。进一步地,多个运算单元和多个寄存器组之间可采用全连接交换网络相连,使得多个运算单元中的任一运算单元能够访问任一寄存器组,因此上述多个寄存器组只需读入和储存一次待卷积数据,无需从存储器多次读取以及存储多次卷积操作中重复的大量数据,因而可以节约多个寄存器组的存储空间,并且减小多个寄存器组和存储器之间的访问带宽的压力,提高了数据访问的并行度。在本申请实施例中,可在多个寄存器组中采用了灵活数据排布方式,可以在一个时钟周期内读出尽可能多的数据,避免了数据读取冲突,实现无阻塞卷积运算过程。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络 单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。
Claims (14)
- 一种计算装置,其特征在于,包括:多个运算单元,用于对多个数据执行多次卷积操作,所述多个运算单元包括第一运算单元和第二运算单元;所述第一运算单元用于:在所述多次卷积操作的第i次卷积操作中,对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作的第i+1次卷积操作中,对所述第一权值数据以及所述第二数据进行乘法运算。
- 如权利要求1所述的装置,其特征在于,还包括:多个寄存器组,用于存储所述多个数据;所述多个运算单元,还用于从所述多个寄存器组中获取所述多次卷积操作中所需的所述多个数据,其中,所述多个运算单元中的每个运算单元能够获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
- 如权利要求2所述的装置,其特征在于,所述第一运算单元,具体用于在所述第i次卷积操作中,从所述多个寄存器组中的第一寄存器组中获取所述第一数据;所述第二运算单元,具体用于在所述第i次卷积操作中,从所述多个寄存器组中的第二寄存器组中获取所述第二数据,其中,所述第一寄存器组和所述第二寄存器组为不同的寄存器组。
- 如权利要求2或3所述的装置,其特征在于,在所述第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中。
- 如权利要求4所述的装置,其特征在于,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第b个寄存器组中,所述多个数据中的第x+a行第s列中的数据存储于第[b+(a*K)]mod(K*K)个寄存器组中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥b≥1,M,N,x,s,b,a为正整数。
- 一种计算方法,其特征在于,应用所述计算方法的计算装置包括多个运算单元,所述多个运算单元用于对多个数据执行多次卷积操作,所述多个运算单元包括第一运算单元和第二运算单元;所述方法包括:在所述多次卷积操作的第i次卷积操作中,所述第一运算单元对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;所述第一运算单元接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作的第i+1次卷积操作中,所述第一运算单元对所述第一权值数据以及所述第二数据进行乘法运算。
- 如权利要求6所述的计算方法,其特征在于,所述计算装置还包括用于存储所述多个数据的多个寄存器组,,其中,所述多个运算单元中的每个运算单元能够获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
- 如权利要求7所述的计算方法,其特征在于,所述方法还包括:在所述第i次卷积操作中,所述第一运算单元从所述多个寄存器组中的第一寄存器组中获取所述第一数据;在所述第i次卷积操作中,所述第二运算单元从所述多个寄存器组中的第二寄存器组中获取所述第二数据,其中,所述第一寄存器组和所述第二寄存器组为不同的寄存器组。
- 如权利要求7或8所述的计算方法,其特征在于:在所述第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中。
- 如权利要求9所述的计算方法,其特征在于,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第r组寄存器中,所述多个数据中的第x+a行第s列中的数据存储于第[r+(a*K)]mod(K*K)组寄存器中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥r≥1,M,N,x,s,r,a为正整数。
- 一种计算装置,其特征在于,包括多个运算单元和多个寄存器组,其中,每个运算单元分别连接所述多个寄存器组;所述多个寄存器组,用于存储待执行多次卷积操作的多个数据;所述多个运算单元,用于从所述多个寄存器组中获取所述多个数据,并对所述多个数据执行多次卷积操作,其中,所述多个运算单元中的每个运算单元用于获取所述多个寄存器组中的任意一个寄存器组中存储的数据。
- 如权利要求11所述的装置,其特征在于,在所述多次卷积操作中的第i次卷积操作过程中,所述多个运算单元从所述多个寄存器组中读取的数据位于所述多个寄存器组的不同寄存器组中,其中,i为大于0的整数。
- 如权利要求11或12所述的装置,其特征在于,所述卷积操作对应的卷积核大小为K*K,所述多个寄存器组包括K*K个寄存器组,所述多个数据包括M行N列数据,所述多个数据在所述K*K个寄存器组中的分布符合以下条件:所述多个数据中的第x行第s列中的数据存储于第b个寄存器组中,所述多个数据中的第x+a行第s列中的数据存储于第[b+(a*K)]mod(K*K)个寄存器组中,其中,K为不小于1的整数,mod表示进行取余运算,M≥x≥1,N≥s≥1,K*K≥b≥1,M,N,x,s,b,a为正整数。
- 如权利要求11至13中任一项所述的装置,其特征在于,所述多个运算单元包括 第一运算单元和第二运算单元,所述第一运算单元用于:在所述多次卷积操作中的第i次卷积操作中,对第一权值数据以及所述多个数据中的第一数据进行乘法运算,其中,i为大于0的整数;接收所述第二运算单元发送的第二数据,其中,所述第二数据是所述第二运算单元在所述第i次卷积操作中执行卷积操作的数据;在所述多次卷积操作中的第i+1次卷积操作中,对所述第一权值数据以及所述第二数据进行乘法运算。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810376471.0A CN110399976B (zh) | 2018-04-25 | 2018-04-25 | 计算装置和计算方法 |
CN201810376471.0 | 2018-04-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019206162A1 true WO2019206162A1 (zh) | 2019-10-31 |
Family
ID=68293769
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/084011 WO2019206162A1 (zh) | 2018-04-25 | 2019-04-24 | 计算装置和计算方法 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110399976B (zh) |
WO (1) | WO2019206162A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113536220A (zh) * | 2020-04-21 | 2021-10-22 | 中科寒武纪科技股份有限公司 | 运算方法、处理器及相关产品 |
CN113807506B (zh) * | 2020-06-11 | 2023-03-24 | 杭州知存智能科技有限公司 | 数据加载电路和方法 |
US11977969B2 (en) | 2020-06-11 | 2024-05-07 | Hangzhou Zhicun Intelligent Technology Co., Ltd. | Data loading |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107679620A (zh) * | 2017-04-19 | 2018-02-09 | 北京深鉴科技有限公司 | 人工神经网络处理装置 |
CN107844826A (zh) * | 2017-10-30 | 2018-03-27 | 中国科学院计算技术研究所 | 神经网络处理单元及包含该处理单元的处理系统 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6023753A (en) * | 1997-06-30 | 2000-02-08 | Billion Of Operations Per Second, Inc. | Manifold array processor |
US8321637B2 (en) * | 2007-05-14 | 2012-11-27 | International Business Machines Corporation | Computing system with optimized support for transactional memory |
CN101237240B (zh) * | 2008-02-26 | 2011-07-20 | 北京海尔集成电路设计有限公司 | 一种利用外部存储器实现卷积交织/解交织的方法及设备 |
CN101841730A (zh) * | 2010-05-28 | 2010-09-22 | 浙江大学 | 一种基于fpga的实时立体视觉实现方法 |
GB2501791B (en) * | 2013-01-24 | 2014-06-11 | Imagination Tech Ltd | Register file having a plurality of sub-register files |
CN103092767B (zh) * | 2013-01-25 | 2016-12-28 | 浪潮电子信息产业股份有限公司 | 一种对云计算内部物理机信息内存池的管理方法 |
CN104035750A (zh) * | 2014-06-11 | 2014-09-10 | 西安电子科技大学 | 一种基于fpga的实时模板卷积实现方法 |
CN104932994B (zh) * | 2015-06-17 | 2018-12-07 | 青岛海信电器股份有限公司 | 一种数据处理方法及装置 |
US10346350B2 (en) * | 2015-10-08 | 2019-07-09 | Via Alliance Semiconductor Co., Ltd. | Direct execution by an execution unit of a micro-operation loaded into an architectural register file by an architectural instruction of a processor |
CN106228240B (zh) * | 2016-07-30 | 2020-09-01 | 复旦大学 | 基于fpga的深度卷积神经网络实现方法 |
CN106844294B (zh) * | 2016-12-29 | 2019-05-03 | 华为机器有限公司 | 卷积运算芯片和通信设备 |
CN106951395B (zh) * | 2017-02-13 | 2018-08-17 | 上海客鹭信息技术有限公司 | 面向压缩卷积神经网络的并行卷积运算方法及装置 |
CN108874445A (zh) * | 2017-10-30 | 2018-11-23 | 上海寒武纪信息科技有限公司 | 神经网络处理器及使用处理器执行向量点积指令的方法 |
-
2018
- 2018-04-25 CN CN201810376471.0A patent/CN110399976B/zh active Active
-
2019
- 2019-04-24 WO PCT/CN2019/084011 patent/WO2019206162A1/zh active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107679620A (zh) * | 2017-04-19 | 2018-02-09 | 北京深鉴科技有限公司 | 人工神经网络处理装置 |
CN107844826A (zh) * | 2017-10-30 | 2018-03-27 | 中国科学院计算技术研究所 | 神经网络处理单元及包含该处理单元的处理系统 |
Also Published As
Publication number | Publication date |
---|---|
CN110399976B (zh) | 2022-04-05 |
CN110399976A (zh) | 2019-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102492477B1 (ko) | 행렬 곱셈기 | |
US11720646B2 (en) | Operation accelerator | |
CN108304922B (zh) | 用于神经网络计算的计算设备和计算方法 | |
US10140251B2 (en) | Processor and method for executing matrix multiplication operation on processor | |
WO2019206162A1 (zh) | 计算装置和计算方法 | |
US10768894B2 (en) | Processor, information processing apparatus and operation method for processor | |
CN114391135A (zh) | 用于对连续分配数据执行存储器内处理操作的方法及相关存储器装置和系统 | |
US7725518B1 (en) | Work-efficient parallel prefix sum algorithm for graphics processing units | |
EP3844610B1 (en) | Method and system for performing parallel computation | |
CN113673701A (zh) | 神经网络模型的运行方法、可读介质和电子设备 | |
US20220107803A1 (en) | Memory device for performing in-memory processing | |
CN110580519A (zh) | 一种卷积运算结构及其方法 | |
CN114579929A (zh) | 加速器执行的方法和电子设备 | |
CN110377874B (zh) | 卷积运算方法及系统 | |
KR20210033757A (ko) | 메모리 장치 및 그 동작 방법 | |
TWI789558B (zh) | 自適應性矩陣乘法器的系統 | |
CN111639701A (zh) | 一种图像特征提取的方法、系统、设备及可读存储介质 | |
US20220129247A1 (en) | Semiconductor device and method of controlling the semiconductor device | |
WO2019206161A1 (zh) | 池化运算装置 | |
JP6906622B2 (ja) | 演算回路および演算方法 | |
CN116010313A (zh) | 一种通用、可配置的图像滤波计算多行输出系统和方法 | |
JP3790060B2 (ja) | 演算処理装置 | |
CN111191774A (zh) | 面向精简卷积神经网络的低代价加速器架构及其处理方法 | |
US11640302B2 (en) | SMID processing unit performing concurrent load/store and ALU operations | |
JP7180751B2 (ja) | ニューラルネットワーク回路 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19793751 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19793751 Country of ref document: EP Kind code of ref document: A1 |