JP6888073B2 - Chip equipment and related products - Google Patents

Description

The present disclosure relates to the fields of communications and chip technology, and in particular to chip devices and related products.

Artificial neural networks (ANN) have been research hotspots in the field of artificial intelligence since the 1980s in the 20th century. It abstracts the human brain neuron network in terms of information processing, establishes a simple model, and forms different networks according to different connection methods. In engineering and academia, it is often referred to directly as neural networks or similar neural networks. A neural network is a computational model consisting of a large number of nodes (or neurons) connected to each other. The calculation of the existing neural network is realized based on the CPU (Central Processing Unit) or the GPU (Graphics Processing Unit), and such a calculation consumes a large amount of power and takes a long calculation time.

Embodiments of the present disclosure provide neural network calculation methods and related products that can reduce calculation time and power consumption of modules.

In a first aspect, embodiments of the present disclosure provide a method of calculating a neural network that is applied in a chip device comprising a main unit and a plurality of basic units. This method involves the following steps: The main unit acquires the calculation schedule data block and the calculation command, and divides the calculation schedule data block into a distribution data block and a broadcast data block according to the calculation command. The main unit divides the distribution data block to obtain a plurality of basic data blocks, distributes the plurality of basic data blocks to the plurality of basic units, and the main unit broadcasts the broadcast data block to the plurality of basic units. The basic unit performs an inner product operation on the basic data block and the broadcast data block, obtains an operation result, and transmits the operation result to the main unit. The main unit processes the calculation result and obtains the command result of the calculation schedule data block and the calculation command.

Optionally, the main unit broadcasts a broadcast data block to multiple basic units, including:

The main unit broadcasts the broadcast data block to a plurality of basic units at one time.

Arbitrarily, the basic unit performs an inner product calculation on the basic data block and the broadcast data block to obtain a calculation result, and transmitting the calculation result to the main unit includes the following.

The basic unit performs inner product processing on the basic data block and the broadcast data block to obtain the result of the inner product processing, accumulates the result of the inner product processing to obtain the calculation result, and transmits the calculation result to the main unit. To do.

Arbitrarily, the calculation result is the result of the inner product processing, the main unit processes the calculation result, and obtaining the command result of the calculation schedule data block and the calculation command includes the following.

The main unit accumulates the calculation results to obtain the accumulation results, arranges the accumulation results, and obtains the command results of the calculation schedule data block and the calculation command.

Optionally, the main unit broadcasts a broadcast data block to a plurality of basic units, including:

The broadcast data block is divided into a plurality of partial broadcast data blocks, and the plurality of partial broadcast data blocks are broadcast to the plurality of basic units several times.

Arbitrarily, the basic unit performs an inner product calculation on the basic data block and the broadcast data block to obtain a calculation result, and transmitting the calculation result to the main unit includes the following.

The basic unit performs inner product processing once on the partial broadcast data block and the basic data block to obtain the result of the inner product processing, and accumulates the results of the inner product processing to obtain the partial calculation result, and the portion thereof. The calculation result is sent to the main unit.

Arbitrarily, the basic unit performs an inner product calculation on the basic data block and the broadcast data block to obtain a calculation result, and transmitting the calculation result to the main unit includes the following.

The basic unit repeatedly uses the partial broadcast data block n times to perform an inner product operation of the partial broadcast data block and n basic data blocks to obtain n partial processing results, and n partial processing results. Is accumulated to obtain n partial calculation results, and the n partial calculation results are transmitted to the main unit, where n is an integer of 2 or more.

In the second aspect, a chip device including a main unit and a plurality of basic units is provided. The basic unit acquires a calculation schedule data block and a calculation command, and divides the calculation schedule data block into a distribution data block and a broadcast data block according to the calculation command. The distribution data block is divided to obtain a plurality of basic data blocks, the plurality of basic data blocks are distributed to a plurality of basic units, and the broadcast data block is broadcast to a plurality of basic units. The basic unit performs an inner product operation on the basic data block and the broadcast data block, obtains an operation result, and transmits the operation result to the main unit. The main unit processes the calculation result and obtains the command result of the calculation schedule data block and the calculation command.

Optionally, the chip device further comprises a branching unit. The branch unit is arranged between the main unit and the basic unit. Branch units are used to transfer data.

Optionally, the main unit is specifically used to broadcast a broadcast data block to multiple base units at once.

Arbitrarily, the basic unit specifically performs inner product processing on the basic data block and the broadcast data block to obtain the result of the inner product processing, and the result of the inner product processing is accumulated to obtain the calculation result. It is used to send the calculation result to the main unit.

Arbitrarily, when the operation result is the result of inner product processing, the main unit accumulates the operation results to obtain the accumulation result, arranges the accumulation results, and commands the calculation schedule data block and the operation command. Get results.

Optionally, the main unit is specifically used to divide a broadcast data block into multiple partial broadcast data blocks and broadcast the multiple partial broadcast data blocks to multiple basic units several times.

Arbitrarily, the basic unit specifically performs inner product processing on the partial broadcast data block and the basic data block to obtain the result of the inner product processing, and accumulates the results of the inner product processing to obtain the partial calculation result. , Used to send the partial calculation result to the main unit.

Arbitrarily, the basic unit specifically uses the partial broadcast data block n times repeatedly to perform an inner product operation of the partial broadcast data block and n basic data blocks, obtains n partial processing results, and n After accumulating each of the partial processing results, it is used to obtain n partial calculation results and send n partial calculation results to the main unit, where n is an integer of 2 or more. ..

Optionally, the main unit comprises one or any combination of main registers or main on-chip cache circuits.

The basic unit comprises one or any combination of basic registers or basic on-chip cache circuits.

Optionally, the main unit comprises one or any combination of a vector operation circuit, an arithmetic logic operation circuit, an accumulator circuit, a matrix transpose circuit, a direct memory access circuit, or a data sorting circuit.

Optionally, the unit comprises one or any combination of an inner product arithmetic circuit, an accumulator circuit, and the like.

Optionally, the branch unit is a plurality of branch units, the main unit is connected to each of the plurality of branch units, and each branch unit is connected to at least one foundation unit.

Optionally, the branch unit is a plurality of branch units, the plurality of branch units are connected in series and then connected to the main unit, and each branch unit is connected to at least one basic unit.

Optionally, the branching unit is specifically used to transfer data between the main unit and the underlying unit.

Optionally, the branching unit is specifically used to transfer data between the main unit and the underlying unit or other branching unit.

Optionally, the data is one or any combination of vectors, matrices, three-dimensional data blocks, four-dimensional data blocks, and n-dimensional data blocks.

If the operation command is a multiplication command by arbitrary selection, it is determined that the multiplier data block is a broadcast data block and the multiplicand data block is a distribution data block.

If the arithmetic command is a convolution command, the input data block is determined to be a broadcast data block and the convolution kernel is determined to be a distribution data block.

A third aspect provides a method of applying the chip device provided by the second aspect. The chip device is used to perform one or any combination of matrix-matrix multiplication, matrix-vector multiplication, convolution operations, or fully connected operations.

In the fourth aspect, a chip that integrates the chip apparatus provided by the second aspect is provided.

A fifth aspect provides a smart device with the chip provided by the sixth aspect.

When the embodiments of the present disclosure are implemented, the following beneficial effects can be obtained.

As can be seen, according to the embodiments of the present disclosure, after receiving the data and the arithmetic command, the data is divided into distribution data and broadcast data, the distribution data is divided into basic data blocks, and then into a plurality of basic units. It is delivered and the internal product calculation is performed. As described above, since the inner product calculation having the largest amount of calculation is distributed to a plurality of basic units and performed at the same time, there is an advantage that the calculation time is shortened and the power consumption is saved.

In order to more clearly exemplify the technical proposals in the embodiments of the present disclosure, the drawings used in the description of the embodiments will be briefly described below. It is clear that the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings based on these drawings without any creative work. ..
FIG. 1a is a schematic structural diagram of the chip device provided by the present disclosure. FIG. 1b is a schematic structural diagram of another chip device provided by the present disclosure. FIG. 1c is a schematic diagram of data distribution of the chip apparatus provided by the present disclosure. FIG. 1d is a schematic diagram of data return of the chip device. FIG. 2 is a schematic flowchart of a neural network calculation method provided by the embodiments of the present disclosure. FIG. 2a is a schematic view of the matrix A provided by the embodiments of the present disclosure multiplied by the matrix B. FIG. 3 is a schematic flowchart of a neural network calculation method provided by the embodiments of the present disclosure. FIG. 3a is a schematic view of a single sample data of full connection 1. FIG. 3b is a schematic view of the multi-sample data of the complete connection 2. FIG. 3c is a schematic diagram of M convolution kernel data of convolution 1. FIG. 3d is a schematic view of the convolution 2 input data. FIG. 3e is a schematic view of the calculation window of the three-dimensional data block of the input data. FIG. 3f is a schematic view of another calculation window of the three-dimensional data block of the input data. FIG. 3g is a schematic view of yet another calculation window of the three-dimensional data block of the input data.

The technical proposal in the embodiments of the present disclosure will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present disclosure, but the embodiments described are part of the embodiments of the present disclosure. Not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present disclosure.

The terms "first", "second", "third", "fourth" and the like described in the specification, claims and drawings of the present disclosure are for distinguishing different objects. It does not explain a particular order. Moreover, the terms "include", "provide" and their variants are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units. Alternatively, optionally, other step units specific to these processes, methods, products or equipment are also included.

"Embodiment" as used herein means that the particular features, structures, or properties described in connection with an embodiment may be included in at least one embodiment of the present disclosure. The appearance of this word in various parts of the specification does not necessarily refer to the same embodiment and is not mutually exclusive, independent or alternative to other embodiments. One of ordinary skill in the art will understand explicitly or implicitly that the embodiments described herein can be combined with other embodiments.

The calculation method of the neural network will be described below using a CPU as an example. Multiplication of matrix and matrix is widely used in neural networks, but here, the logical product operation in the CPU will be described by taking the multiplication of matrix A and matrix B as an example. As shown below, it is assumed that the result of the logical product of the matrix A and the matrix B is C, that is, C = A * B.

Therefore, in the CPU or GPU, it is necessary to calculate line by line, after the calculation of the first line is completed, the calculation of the second line is performed, and the calculation of the third line is performed until the calculation of all lines is completed. .. In a neural network, the calculation time is very long because the data may be thousands of rows. During the calculation, the CPU is in the operating state for a long time, and the energy consumption is also large.

See FIG. 1b. FIG. 1b is a schematic structural diagram of a chip device, and as shown in FIG. 1b, the chip device includes a main unit circuit, a basic unit circuit, and a branch unit circuit. The main unit circuit includes a register and / or an on-chip cache circuit. The main unit is one of a vector arithmetic unit, an ALU (Arithmetic and Logic Unit) circuit, an accumulator circuit, a matrix transfer circuit, a DMA (Direct Memory Access) circuit, a data sorting circuit, and the like. It may be provided with one or any combination. Each foundation unit may include a base register and / or a base-on-chip cache circuit. Each foundation unit may further include one or any combination of an inner product arithmetic circuit, a vector arithmetic circuit, an accumulator circuit, and the like. All the circuits may be integrated circuits. If a branch unit exists, the main unit is connected to the branch unit and the branch unit is connected to the basic unit. The basic unit performs an internal product operation between data blocks, the main unit sends and receives external data, and the external data is distributed to the branch unit. The branch unit is used to send and receive data from the main unit or the basic unit. As for the main unit, since the number of connected units is limited, it is complicated because it is necessary to add a branch unit between the main unit and the basic unit in order to realize more access of the basic unit. Realize the calculation of various data blocks. Therefore, the structure shown in FIG. 1b is suitable for calculating complex data.

The connection structure between the branch unit and the foundation unit may be arbitrary and is not limited to the H-shaped structure shown in FIG. 1b. Arbitrarily, from the main unit to the basic unit is a broadcast or distribution configuration, and from the basic unit to the main unit is a gather configuration. The definitions of broadcast, delivery and collection are as follows:

The data transmission method from the main unit to the basic unit may include the following.

The main unit is connected to a plurality of branch units, and each branch unit is connected to a plurality of basic units.

The main unit is connected to one branch unit, the branch unit is connected to one branch unit, and so on, multiple branch units are connected in series, and each branch unit is connected to multiple foundation units. To.

The main unit is connected to each of a plurality of branch units, and each branch unit is connected in series with a plurality of basic units.

The main unit is connected to one branch unit, the branch unit is connected to one branch unit, and so on, multiple branch units are connected in series, and each branch unit is connected in series with multiple foundation units. Be connected.

When delivering data, the main unit sends data to some or all of the underlying units, and the data received by each underlying unit that receives the data may be different.

When broadcasting data, the main unit sends the data to some or all of the underlying units, and each underlying unit that receives the data receives the same data.

When data is collected, some or all of the underlying units send the data to the main unit. It should be noted that the chip device shown in FIGS. 1a or 1b may be an independent physical chip, but of course in practical applications the chip device is integrated into another chip (eg CPU, GPU). May be done. Embodiments of the present invention do not limit the physical representation of the chip device described above.

See FIG. 1c. FIG. 1c is a schematic diagram of data distribution of the chip device. As shown by the arrow in FIG. 1c, the arrow indicates the data distribution direction. As shown in FIG. 1c, after the main unit receives the external data, the external data is divided and then distributed to a plurality of branch units, and the branch unit transmits the divided data to the main unit.

See FIG. 1d. FIG. 1d is a schematic diagram of data return of the chip device. As shown by the arrow in FIG. 1d, the arrow is the data return direction. As shown in FIG. 1d, the basic unit returns data (for example, the result of inner product calculation) to the branch unit, and the branch unit returns it to the main unit.

See FIG. 1a. FIG. 1a is a schematic structural diagram of another chip device. The chip device includes a main unit and a basic unit, and the main unit is connected to the basic unit. In the structure shown in FIG. 1a, since the basic unit is directly physically connected to the main unit, the number of basic units to be connected is limited, and the structure is suitable for simple data calculation.

See FIG. FIG. 2 provides a method of calculating a neural network using the above chip device. The method is performed using a chip device as shown in FIG. 1a or FIG. 1b. The method includes the following steps, as shown in FIG.

In step S201, the main unit of the chip device acquires the calculation schedule data block and the calculation command.

The data block to be calculated in step S201 may be specifically a matrix, a vector, three-dimensional data, four-dimensional data, multidimensional data, or the like, and the present disclosure expresses a specific representation of the above-mentioned data block. Not particularly limited. The arithmetic command may be specifically a multiplication command, a convolution command, an addition command, a subtraction command, a BLAS (English: Basic Linear Algebra Subprograms) function, an activation function, or the like.

In step S202, the main unit divides the calculation schedule data block into the distribution data block and the broadcast data block in response to the calculation command.

Specifically, the method of carrying out the above-mentioned step S202 is as follows.

If the arithmetic command is a multiplication command, it is determined that the multiplier data block is a broadcast data block and the multiplicand data block is a distribution data block.

If the arithmetic command is a convolution command, the input data block is determined to be a broadcast data block and the convolution kernel is determined to be a distribution data block.

In step S2031, the main unit divides the distribution data block to obtain a plurality of basic data blocks, and distributes the plurality of basic data blocks to the plurality of basic units.

In step S2032, the main unit broadcasts the broadcast data block to the plurality of basic units.

Arbitrarily, the above-mentioned steps S2031 and S2032 may be repeatedly executed. When the amount of data is relatively large, the main unit divides the distribution data block to obtain multiple basic data blocks, divides each basic data block into m basic data subblocks, and broadcast data blocks are also m. Divide into broadcast data subblocks. The main unit delivers one basic data subblock each time and broadcasts one broadcast data subblock. The basic data subblock and the broadcast data subblock are executable data blocks of parallel neural network calculation. For example, for example, when the matrix A of 1000 * 1000 is multiplied by the matrix B of 1000 * 1000, the basic data block is the data in the z-th row of the matrix A, and the basic data subblock is the z-th row of the matrix A. The data in the first 20 columns of the data of the above, and the broadcast data subblock may be the data in the first 20 rows of the z-th column of the matrix B.

The basic data block in step S203 may be the smallest data block that can be specifically subjected to the inner product calculation. Taking matrix multiplication as an example, the basic data block may be one row of data in the matrix. For example, in the case of a convolution operation, the basic data block may be the weight of the convolution kernel.

Regarding the delivery method in step S203 described above, the following description of the embodiment may be referred to, and no verbosity is given here. Regarding the method of broadcasting the broadcast data block, the following description of the embodiment may be referred to, and no verbosity is given here.

In step S2041, the basic unit of the chip device performs an inner product calculation on the basic data block and the broadcast data block, and obtains a calculation result (which may be an intermediate result).

In step S2042, if the calculation result is not an intermediate result, the calculation result is returned to the main unit.

Regarding the return method in step S204, the following description of the embodiment may be referred to, and no verbosity is given here.

In step S205, the main unit processes the calculation result and obtains the calculation schedule data block and the command result of the calculation command.

The processing method of step S205 may be accumulation, array, or the like, and the present disclosure is not limited to a specific method of the above processing, and the specific method includes, for example, a non-linear transformation, and responds to different arithmetic commands. It may be configured.

In the technical proposal provided by the present disclosure, when performing an operation, the main unit receives external data including the calculation schedule data block and the calculation command, acquires the calculation schedule data block and the calculation command, and calculates the calculation schedule data according to the calculation command. Determine the distribution data block and broadcast data block of the block, divide the distribution data block into multiple basic data blocks, broadcast the broadcast data block to multiple basic units, and distribute the multiple basic data blocks to multiple basic units. To do. Each of the plurality of basic units performs an inner product calculation on the basic data block and the broadcast data block, and obtains the calculation result. The plurality of basic units return the operation result to the main unit, and the main unit obtains the command result of the operation command according to the returned operation result. The technical points of this technical proposal are as follows. With respect to the neural network, a large amount of operations are in the inner product operation between the data blocks, the overhead of the inner product operation is large, and the calculation time is long. Therefore, in the embodiment of the present disclosure, the operation command and the operation schedule command are used. First, distinguish between the distribution data block and the broadcast data block in the calculation schedule data block. The broadcast data block is a data block that must be used when performing the inner product operation, and the distribution data block is a data block that can be divided in the inner product operation. Taking matrix multiplication as an example, for example, the calculation schedule data blocks are matrix A and matrix B, the arithmetic command is a matrix multiplication command (A * B), and in matrix multiplication, the multiplicand matrix A is a plurality of basic data blocks. Since the multiplier matrix B may be broadcast, it is determined that matrix A is a divisible data block and matrix B is a broadcast data block according to the rules of matrix multiplication. According to the definition of matrix multiplication, the data in each row of the multiplicand matrix A needs to perform an inner product operation with the multiplier matrix B, respectively. Therefore, in the technical proposal of the present application, the matrix A is divided into M basic data blocks. In the M basic data blocks, each basic data block may be one row of data in the matrix A. Therefore, in matrix multiplication, the time-consuming calculation time is performed by each of the plurality of basic units, and in the inner product calculation, the results can be calculated in parallel at high speed by the plurality of basic units, so that the calculation time can be shortened. The shorter calculation time can also reduce the operating time of the chip device, thereby reducing power consumption.

The effects of the proposed technology provided by the present disclosure are described below through practical examples. As shown in FIG. 2a, it is a schematic diagram which multiplies the matrix A by the vector B. As shown in FIG. 2a, the matrix A has M rows and L columns, and the vector B has L rows. Assuming that the time required for the calculator to calculate the inner product of one row of matrix A and vector B is t1, when calculating with the CPU or GPU, it is necessary to calculate the next row after completing the calculation of one row. Yes, in the method of calculating with GPU or CPU, the calculation time is T0 = m * t1. According to the proposed technology provided by the embodiments of the present disclosure, assuming that the matrix A has M basic units, the matrix A is divided into M basic data blocks, and each basic data block is one row of the matrix A. When it is row data and M basic units perform inner product operations at the same time, the calculation time is t1, and the time required to adopt the technical proposal provided by the embodiment of the present disclosure is T1 = t1 + t2 + t3. .. Here, t2 is the time required for the main unit to divide the data, and t3 is the time required to process the calculation result of the inner product operation and obtain the command result. Since the amount of calculation for processing the data division and the calculation result is very small, the time required is very short, so T0 >> T1. Therefore, adopting the technical proposal of the embodiment of the present disclosure can significantly reduce the calculation time. At the same time, with respect to the power consumption of the scheduled calculation data, the chip apparatus provided by the present disclosure has a short working time according to T0 >> T1. Experiments have shown that when the working time of a chip device is very short, its energy consumption is much less than the long working time, which has the advantage of saving energy.

In step S203, there are many ways in which the main unit can broadcast a broadcast data block to a plurality of basic units. Specifically, it is as follows.

Method A: A broadcast data block is broadcast to a plurality of basic units at once. (Broadcast refers to "one-to-many" data transmission, that is, the main unit transmitting the same block of data to multiple (all or part) underlying units at the same time). For example, in Matrix A * Matrix B, Matrix B is a broadcast data block that broadcasts Matrix B to a plurality of basic units at once. In another example, in convolution, the input data is a broadcast data block, which broadcasts the input data block to multiple basic units at once. The advantage of this method is that the amount of data transmitted between the main unit and the basic unit can be saved. That is, all broadcast data can be transmitted to a plurality of basic units in one broadcast.

Method B: The broadcast data block is divided into a plurality of partial broadcast data blocks, and the plurality of partial broadcast data blocks are broadcast to a plurality of basic units, for example, several times. For example, Matrix B is broadcast to a plurality of basic units several times. Specifically, the data in column N of matrix B is broadcast each time. The advantage of this method is that the configuration of the basic unit can be reduced. The storage capacity of the registers placed in the basic unit is not large. In the case of the matrix B having a large amount of data, if the matrix B is distributed to the basic unit at one time, a relatively large register capacity is required to store these data. Since the number of basic units is large, increasing the register capacity inevitably has a large effect on cost increase, so a method of broadcasting a broadcast data block several times is adopted here. That is, the base unit only needs to store a portion of the data in the broadcast data block that is broadcast each time, thereby reducing costs.

What I would like to explain is that the above-mentioned method A or method B may be adopted for delivering the plurality of basic data blocks to the plurality of basic units in the above-mentioned step S203. The only difference is that the transmission method is a unicast method and the data to be transmitted is a basic data block.

Specifically, the method of carrying out the above-mentioned step S204 is as follows.

When broadcasting a broadcast data block by method A and delivering a basic data block by method A (as shown in FIG. 3a), the basic unit performs inner product processing on the basic data block and the broadcast data block, and performs inner product processing. Get results. That is, one line of inner product calculation is performed at a time, the result of the inner product processing (one of the calculation results) is transmitted to the main unit, and the main unit accumulates the result of the inner product processing. Of course, in an actual application, after the basic unit accumulates the result of the inner product processing, the accumulated result (another operation result) is transmitted to the main unit. According to the above method, the amount of data transmitted between the main unit and the basic unit can be reduced and the calculation speed can be improved.

When the broadcast data block of method B is adopted, in the optional technical proposal, each time the basic unit receives the partial broadcast data block, the basic unit performs a partial internal product operation between the basic data block and the partial broadcast data block. The processing result is transmitted to the main unit, and the main unit accumulates the processing result. In another optional option, when the basic unit receives n basic data blocks, the broadcast data block is repeatedly used, and the inner product calculation of the broadcast data block and n basic data blocks is performed, and n basic data blocks are calculated. Obtain the partial processing result. The basic unit transmits the n processing results to the main unit, and the main unit accumulates the n processing results. Of course, the above accumulation can also be performed in the basic unit.

In the above case, the amount of data in the broadcast data block is generally very large, and the distribution data block is also large. Since chip devices belong to the hardware configuration, the number of basic units to be placed is theoretically infinite, but in reality, the number is limited, and generally dozens of basic units are used. Is. With the development of technology, the number can change constantly, for example by increasing. However, in the matrix-matrix multiplication in the neural network, the number of rows in the matrix A is several thousand, and the number of columns in the matrix B is also several thousand. Therefore, the matrix B is transmitted to the basic unit with one broadcast data. It is not feasible to do. Then, a realization method may be adopted in which a part of the data of the matrix B, for example, the data of the first five columns is broadcast each time. A similar method may be adopted for the matrix A. For the basic unit, the partial dot product calculation may be performed each time, and the result of the partial dot product calculation is stored in the register, and after all the dot product operations of that row are executed, the results of all the dot product operations of that row are calculated. Accumulate to obtain one type of calculation result, and send the calculation result to the main unit. This method has the advantage of improving the calculation speed.

See FIG. FIG. 3 provides a method for calculating a neural network. The calculation in this embodiment is described by the calculation method of Matrix A * Matrix B. Matrix A * Matrix B may be a schematic diagram of the matrix shown in FIG. 3a. For convenience of explanation, the calculation method of the neural network shown in FIG. 3 is performed by the chip apparatus shown in FIG. 1b. As shown in FIG. 1b, the chip device has 16 basic units. Here, for convenience of explanation and allocation, the value of M may be set to 32, the value of N may be set to 15, and the value of L may be set to 20. Of course, it should be understood that the arithmetic unit may have any number of basic units. This method includes the following steps, as shown in FIG.

In step S301, the main unit receives the matrix A, the matrix B, and the multiplication operation command A * B.

In step S302, the main unit determines that the matrix B is a broadcast data block and the matrix A is a distribution data block according to the multiplication operation command A * B, divides the matrix A into 32 basic data blocks, and divides the matrix A into 32 basic data blocks. Each basic data block is one row of data in Matrix A.

In step S303, the main unit equally allocates 32 basic data blocks to 16 basic units, and evenly allocates 32 basic data blocks to 16 basic units, that is, 2 for each basic unit. Is to receive the basic data block of. The allocation method of these two data blocks may be any non-repeating allocation order.

As the allocation method in step S303, some other allocation method may be adopted. For example, if the number of data blocks cannot be evenly distributed to each foundation unit, the database may be unevenly allocated to each foundation unit. A method such as dividing some non-equally allocated data blocks and then evenly allocating them may be used. The embodiments of the present disclosure do not limit how the basic data blocks described above are assigned to a plurality of basic units.

In step S304, the main unit extracts the partial data of the first sequence of matrix B (for example, the first 5 columns) and broadcasts the partial data of the first 5 columns of matrix B to 16 basic units.

In step S305, the 16 basic units repeatedly use the partial data of the first 5 columns to perform inner product calculation and cumulative calculation with the two basic data blocks, obtain 32 * 5 preprocessing results, and obtain 32 * 5 preprocessing results. * Send 5 pre-processing results to the main unit.

In step S306, the main unit extracts the partial data of the five columns of the matrix B and broadcasts the partial data of the five columns of the matrix B to the 16 basic units.

In step S307, the 16 basic units repeatedly use the partial data in the central 5 columns to perform the inner product calculation and the cumulative calculation with the 2 basic data blocks, and obtain 32 * 5 intermediate processing results. The 32 * 5 intermediate processing results are transmitted to the main unit.

In step S308, the main unit extracts the partial data of the last 5 columns of the matrix B and broadcasts the partial data of the last 5 columns of the matrix B to the 16 basic units.

In step S309, the 16 basic units repeatedly use the last 5 columns of partial data to perform inner product operations and cumulative operations with the 2 basic data blocks to obtain 32 * 5 post-processing results. 32 * 5 post-processing results are transmitted to the main unit.

In step S310, the main unit combines 32 * 5 pretreatment results, 32 * 5 intermediate treatment results, and 32 * 5 posttreatment results according to the pre-, middle, and post-processes to form a 32 * 15 matrix. Get C. This matrix C is the command result of matrix A * matrix B.

In the proposed technique shown in FIG. 3, the matrix A is divided into 32 basic data blocks, and then the matrix B is broadcast in batch, so that the basic unit can obtain the command result in batch. Since the inner product is divided into 16 basic units for calculation, there are advantages that the calculation time is significantly shortened, the calculation time is short, and the energy consumption is low.

See FIG. 1a. FIG. 1a is a chip device provided by the present disclosure. The chip device includes a main unit and a basic unit, the main unit is a hardware chip device, and the basic unit is also a hardware chip device.

The main unit is used to perform each continuous operation in the neural network operation and data transmission with the basic unit.

The basic unit is used to perform parallel accelerated operations in a neural network based on the data transmitted by the main unit and transmit the operation results to the main unit.

The parallel-accelerated operations include, but are not limited to, large-scale parallelizable operations such as multiplication operations between data blocks and convolution operations.

Each of the above continuous operations includes, but is not limited to, continuous operations such as cumulative operations, matrix transpose operations, and data array operations.

For the main unit and a plurality of basic units, the main unit acquires a calculation schedule data block and a calculation command, and divides the calculation schedule data block into a distribution data block and a broadcast data block according to the calculation command. The distribution data block is divided to obtain a plurality of basic data blocks, the plurality of basic data blocks are distributed to a plurality of basic units, and the broadcast data block is broadcast to a plurality of basic units. The basic unit performs an inner product operation on the basic data block and the broadcast data block, obtains an operation result, and transmits the operation result to the main unit. The main unit processes the calculation result and obtains a command result of a calculation schedule data block and a calculation command.

Optionally, the chip device further comprises a branching unit, which is located between the main unit and the basic unit and is used to transfer data.

Optionally, the main unit is specifically used to broadcast a broadcast data block to multiple basic units at once.

Arbitrarily, the basic unit specifically performs inner product processing on the basic data block and the broadcast data block to obtain the result of the inner product processing, and the result of the inner product processing is accumulated to obtain the calculation result. It is used to send the calculation result to the main unit.

Arbitrarily, when the operation result is the result of the internal product processing, the main unit accumulates the operation results to obtain the accumulation result, arranges the accumulation results, and commands the calculation schedule data block and the operation command. Is used to obtain.

Optionally, the main unit is specifically used to divide the broadcast data block into multiple partial broadcast data blocks and broadcast the multiple partial broadcast data blocks to multiple basic units several times.

Arbitrarily, the basic unit specifically performs the inner product processing once for the partial broadcast data block and the basic data block to obtain the result of the inner product processing, and accumulates the results of the inner product processing to obtain the partial calculation result. It is used to send the partial calculation result to the main unit.

Arbitrarily, the basic unit specifically uses the partial broadcast data block n times repeatedly to perform an inner product operation between the partial broadcast data block and n basic data blocks, and obtains n partial processing results. After accumulating n partial processing results, n partial calculation results are obtained and n partial calculation results are transmitted to the main unit. Here, n is an integer of 2 or more.

The embodiments of the present disclosure further provide a method of applying a chip device as shown in FIG. 1a. The method can be specifically used to perform one or any combination of matrix-matrix multiplication operations, matrix-vector multiplication operations, convolution operations, or fully connected operations.

Specifically, the main unit may perform neural network calculation steps such as pooling operation, regularization (normalization) operation, for example, batch normalization (batch normalization), lrn, and the like.

An embodiment of the present application also provides a chip with a chip device as shown in FIG. 1a or FIG. 1b.

An embodiment of the present application further provides a smart device including the above-mentioned chip, in which a chip device as shown in FIG. 1a or FIG. 1b is integrated. Smart devices include, but are not limited to, smartphones, tablet computers, personal digital assistants, smart watches, smart cameras, smart TVs, smart refrigerators, and the like. The above-mentioned devices are for illustrative purposes only, and embodiments of the present application are not limited to the particular form of the above-mentioned devices.

The calculation of the matrix multiplied by the above may refer to the description of the embodiment shown in FIG. 3, and is not verbose here.

Perform a complete connection calculation using a chip device.

The input data of the fully connected layer is a vector of length L (eg, vector B of "fully connected 1-single sample" shown in FIG. 3a) (ie, the input of the neural network is a single sample) and is fully connected. If the output of the layer is a vector of length M and the weight of the fully connected layer is an M * L matrix (eg, matrix A of "FIG. 3b fully connected 1-single sample"), the weight matrix of the fully connected layer. Is a matrix A (that is, a divided data block), and the input data is a vector B (that is, a broadcast data block), and the calculation is performed according to the method 1 shown in FIG. The specific calculation method may be as follows.

The input data of the fully connected layer is a matrix (that is, when the input of the neural network performs batch operations on multiple samples) (the input data of the fully connected layer represents N input samples, and each sample is long. It is a vector of L, and the input data is represented by a matrix of L * N, for example, represented by matrix B of "Fig. 3b fully connected 1-multiple samples"), and the output of the fully connected layer for each sample is long. In the case of a vector of M, the output data of the fully connected layer is a matrix of M * N like the result matrix of "Fig. 3a fully connected-1 multiple samples", and the weight of the fully connected layer is M * L. It is a matrix (eg, Matrix A in "Fig. 3a Fully Connected-1 Multiple Samples"), where the weighted matrix of the fully connected layer is Matrix A (ie divided data block) and the input data matrix is Matrix B (ie broadcast data block) Or, the weight matrix of the fully connected layer is set to matrix B (broadcast data block), and the input vector is set to matrix A (divided data block), and the calculation is performed according to the method 1 shown in FIG.

Chip device When performing artificial neural network operations using a chip device, the convolution layer, pooling layer, and normalization layer (also called standardization layer, for example, BN (Batch normalization) or LRN (Local Response Normalization)) in the neural network )) And other input data are shown in "Fig. 3d Convolution 2-Input Data" (for clarity, the 3D data blocks representing each sample are, for example, C = 5, H = 10, W = Although described with reference to 12, in actual use, the sizes of N, C, H, and W are not limited to the values shown in FIG. 3d), and each three-dimensional data block of FIG. 3d has one sample. Indicates that it corresponds to the input data of the layer. The three dimensions of each three-dimensional data block are C, H, and W, respectively, and there are N such three-dimensional data blocks.

When performing the calculation of the neural network layer, after receiving the input data, the main unit arranges the input data in a certain order for each input data sample by using the data sorting circuit of the main unit. The order may be any order.

Arbitrarily, the order arranges the input data in a manner such as NHWC and NWHC, in which the C-dimensional coordinates shown in the above schematic change are the fastest. Among them, C is the dimension of the innermost layer of the data block, N is the dimension of the outermost layer of the data block, and H and W are the dimensions of the intermediate layer. The result of this is that the data in C are adjacent, which makes it easier to improve the degree of parallelism of the operations and makes it easier to perform parallel operations on multiple feature maps.

The following describes how C, H, and W are understood for different neural network operations. In the case of convolution and pooling, H and W are the sliding dimensions of the associated calculation window when performing the convolution and pooling operations (an example of the calculation window sliding in the W dimension is "Convolution 3-" in Fig. 3e. It is shown in two figures, "Slide A" and "Convolution 3-Slide B" in FIG. 3f, and it is shown, for example, in FIG. 3g that the calculation window slides in the H dimension). The size of the arithmetic window matches the size of one convolution kernel of M convolution kernels. There are M convolution kernels as shown in FIG. 3c, each convolution kernel is a 5 * 3 * 3 3D data block, and its calculation window is also a 5 * 3 * 3 3D data block. The KH and KW of the M convolution kernels shown in FIG. 3c indicate that the dimension corresponding to KH is the H dimension of the input data and the dimension corresponding to KW is the W dimension of the input data. The gray cubes in FIGS. 3e, 3f, and 3g are the data used to slide the calculation window each time to perform the calculation. The slide direction may be H as the slide direction and then W as the slide direction, or W may be the slide direction and then H may be the slide direction. Specifically, in the case of convolution, the operations in each sliding window are the data block represented by the cube in the gray part of the figure and the M convolution kernel data blocks represented by "Convolution 1-convolution kernel in Fig. 3c". It is an inner product operation performed on each of. In convolution, the value corresponding to each convolution kernel is output for each sliding window position. In the case of pooling, the calculation in each sliding window is that the data blocks represented by the gray cubes in the figure are in H and W dimensions (in the example of the figure, nine of the gray data blocks on the same plane). Operations such as selecting the maximum value (of the numerical values) or calculating the average value. In pooling, C values are output for each position of the sliding window. C is another dimension other than H and W in a single sample 3D data block. N represents that the operations of this layer are executed simultaneously with a total of N samples. In the LRN of the regularization algorithm, the definition of C dimension is as follows. In each basic LRN operation, one continuous data block (that is, a Y * 1 * 1 data block) is selected along the C dimension. Here, Y of the data block of Y * 1 * 1 is a value in the C dimension, the value of Y is equal to or less than the maximum value in the C dimension, the first 1 represents the H dimension, and the second 1 is W. Represents a dimension. The remaining two dimensions are defined as H and W dimensions, that is, for each of the 3D data blocks of each sample, each time an LRN regularization operation is performed, the same W coordinate and the same H coordinate but consecutive in different C coordinates. Performs operations on some target data. In the case of the regularization algorithm BN, the mean and variance (or standard deviation) are calculated for all the numerical values having the same C-dimensional coordinates in the 3D data block of N samples.

In "FIGS. 3c to 3g", one cube is used to represent one numerical value, which is also called a weight. The numbers used in the schematic are limited to examples. In the actual case, the dimensional data can be any numerical value (including the case where a certain dimension is 1, in which case the 4D data block automatically becomes a 3D data block. For example, the number of samples calculated at the same time. If is 1, the input data is a 3D data block. In another example, if the number of convolution kernels is 1, the convolution and data are 3D data blocks). A convolution operation is performed between the input data B and the convolution kernel A using a chip device.

In the case of a convolution layer, its weight (all convolution kernels) is as shown in "Fig. 3c Convolution 1-Convolution Kernel", the number of convolution kernels is M, and each convolution kernel is from a matrix of C KH rows and KW columns. Since it is configured, the weight of the convolution layer can be represented as a four-dimensional data block in which the four dimensions are M, C, KH, and KW, respectively. As shown in "Fig. 3d Convolution 2-Input Data", the input data of the convolution layer is a 4-dimensional data block consisting of N 3D data blocks, and each 3D data block has C H rows and W columns. Consists of a feature matrix of (ie, the four dimensions are N, C, H, W data blocks, respectively). The weights of each of the M convolution kernels are distributed from the main unit to one of the K base units and stored in the base unit's on-chip cache and / or register (at this point M). Each convolution kernel is a distribution data block, and each convolution kernel may be a base data block, but of course, in practical applications, the base data block is more like one plane matrix of one convolution kernel. It may be changed to a smaller temperature). The specific delivery method may be as follows. When the number of convolution kernels M ≤ K, the weight of one convolution kernel is distributed to each of the M basic units. When the number of convolution kernels M> K, the weights of one or more convolution kernels are distributed to each base unit (the convolution kernel weight set delivered to the i-th base unit is Ai, for a total of Mi). Has a convolution kernel). In each base unit, for example in the i-th base unit, the convolution kernel weight Ai received by the main unit is stored in its registers and / or on-chip cache, and each part of the input data (ie, FIG. 3e, (Sliding window as shown in FIG. 3f or 3g) is transmitted to each basic unit by a broadcast method (the above-mentioned broadcasting method may adopt the above-mentioned method A or method B), and the number is used for broadcasting. The weights of the arithmetic window can be broadcast to all the basic units by the method of broadcasting once, and specifically, the weights of some arithmetic windows may be broadcast each time, for example, a matrix of one plane is broadcast each time. However, taking FIG. 3e as an example, the KH * KW matrix in the C plane can be broadcast each time, and of course in practical applications, the first n rows or the first n columns of the KH * HW matrix in the C plane. The data can be broadcast at one time, and the present disclosure does not limit the method of transmitting the partial data and the method of arranging the partial data. The input data is arranged in an arbitrary dimensional order, and then each part of the input data is sequentially broadcast to the main unit in order. Arbitrarily, the transmission method of the convolution kernel, which is the distribution data described above, may also adopt a method transmission method similar to the input data calculation window, and is not exaggerated here. Arbitrarily, C is converted to the innermost loop in the arrangement of the input data. As a result of this, since the data of C becomes adjacent, the degree of parallelism of the convolution operation is improved, and it becomes easier to execute the parallel operation in a plurality of feature maps. Arbitrarily, the arrangement of the input data is converted to the arrangement in which the dimensional order is NHWC or NWHC. Each foundation unit, eg, the i-th foundation unit, calculates the inner product of the convolution kernel at weight Ai and the corresponding portion of the received broadcast data (ie, the arithmetic window). The data in the corresponding portion of the weight Ai can be read directly from the on-chip cache and used, or first read into a register for repeated use. The result of the inner product operation of each foundation unit is accumulated and transmitted back to the main unit. The partial sum obtained by executing the inner product operation by the basic unit is transmitted to the main unit and returned to the main unit for accumulation. The partial sum obtained by performing the inner product operation by the basic unit is stored in the register and / or on-chip cache of the basic unit each time, and is returned to the main unit after the accumulation is completed. The partial sum obtained by performing the inner product operation of each basic unit is accumulated by storing the partial sum in the register and / or on-chip cache of the basic unit in some cases, and in some cases, it is transmitted to the main unit and accumulated. Calculate and return to the main unit after the accumulation is completed.

How to implement a BLAS (Basic Linear Algebra Subprograms) function using a chip device GEMM, GEMM calculation refers to the operation of matrix-matrix multiplication in the BLAS library. The usual representation of this operation is: C = alpha * op (A) * op (B) + beta * C. Where A and B represent two inputs, C is an output matrix, alpha and beta are scalars, and op is an operation on matrix A or B. In addition, there are some auxiliary integers as parameters to explain the width and height of the matrices A and B.

The steps to realize the GEMM calculation using the device are as follows.

Each op operation is performed on the input matrix A and the matrix B. The op operation may be a matrix transposition operation, but of course, it may be another operation such as a nonlinear function operation or pooling. The matrix up operation is realized by using the vector calculation function of the main unit. If the op of a matrix may be empty, the main unit does not perform any operation on that matrix.

The matrix multiplication between op (A) and op (B) is performed by the method shown in FIG.

Using the vector calculation function of the main unit, the operation of multiplying each value in the result of op (A) * op (B) by alpha is performed.

Using the vector calculation function of the main unit, the addition step of the corresponding positions of the matrix alpha * op (A) * op (B) and beta * C is realized.

GEMV
GEMV calculation refers to the operation of matrix-vector multiplication in the BLAS library. The usual representation of this operation is: C = alpha * op (A) * B + beta * C. Here, A is an input matrix, B is an input vector, C is an output vector, alpha and beta are scalars, and op is an operation on matrix A.

The steps to realize the GEMV calculation using the device are as follows.

Performing the corresponding op operation on the input matrix A, the chip device completes the matrix-vector multiplication between the matrix op (A) and the vector B using the method shown in FIG. Using the vector calculation function of the main unit, the operation of multiplying each value of the result of op (A) * B by alpha is performed. The vector operation function of the main unit is used to realize the addition step of the corresponding positions between the matrices alpha * op (A) * B and beta * C.

How to Realize an Activation Function Using a Chip Device An activation function usually refers to performing a non-linear operation on each piece of data in one data block (which may be a vector or a multidimensional matrix). For example, the activation function may be y = max (m, x), where x is an input value, y is an output value, and m is a constant. The activation function may be y = tanh (x), where x is the input value, y = sigmoid (x). The activation function may be a piecewise linear function. The activation function may be any function that inputs one data and outputs one data.

When realizing the activation function, the chip device inputs a vector and calculates the activation vector of the vector by using the vector calculation function of the main unit. The main unit applies an activation function to each value of the input vector (the input of the activation function is also one numerical value and the output is also one numerical value), and the corresponding position from one numerical output to the output vector is determined. calculate.

The source of the input vector includes, but is not limited to, external data of the chip device and calculation result data of the basic unit transferred by the branch unit of the chip device.

Specifically, the calculation result data may be a calculation result of multiplying a matrix by a vector. More specifically, the calculation result data may be a calculation result obtained by multiplying a matrix by a matrix. The input data may be the calculation result after the main unit realizes that the offset is applied.

Realizing the operation of applying offset using a chip device The addition function of two vectors or two matrices can be realized by the main unit. The main unit can provide the ability to add a vector to each row of the matrix or to each column.

Optionally, the matrix may be from the result of the device performing matrix-matrix multiplication operations. The matrix may be the result of the device performing an operation that multiplies the matrix by a vector. The matrix may be data received from the outside by the main unit of the device. The vector may be data received from the outside by the main unit of the device.

The above input data and calculation result data are merely examples, and data from other types and sources are also possible in actual applications, and the embodiments of the present disclosure limit the sources and representation methods of the above data. It's not a thing.

In embodiments of the methods described above, for brevity, they are all described as a combination of actions, but those skilled in the art will appreciate that the present disclosure is not limited by the sequence of actions described. This is because certain steps can be performed in other sequences or at the same time in accordance with the present disclosure. Further, one of ordinary skill in the art will appreciate that the embodiments described herein are optional embodiments and that relevant actions and modules are not necessarily required by the present disclosure.

In the above embodiments, the description of the various embodiments has its own emphasis, and the parts not detailed in one embodiment may refer to the relevant description of the other embodiments.

It should be understood that in some embodiments provided herein, the disclosed devices may also be implemented in other ways. For example, the embodiment of the device described above is merely an example, for example, the division of units is only the division of logical functions, and when actually implemented, other division methods, for example, a combination of a plurality of units or components are combined. It may be integrated into another system, or some features may or may not be ignored. Further, the mutual coupling or direct coupling or communicable connection illustrated or described may be an indirect coupling or communicable connection via any interface, device or unit, and may be electrical or other.

Further, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, physically separated, or two or more devices may be integrated. The above integrated units / modules are implemented in the form of hardware. For example, the hardware may be a circuit including a digital circuit, an analog circuit, and the like. The physical realization of the hardware structure includes, but is not limited to, physical devices, and physical devices include, but are not limited to, transistors, memristors, and the like. The computing module in the computing device may be any suitable hardware processor such as CPU, GPU, FPGA, DSP, ASIC. The storage device may be any suitable magnetic storage medium or optical magnetic storage medium such as RRAM®, DRAM, SRAM, EDRAM, HBM, HMC.

The described units may or may not be physically separated, i.e. they may be located in one place or distributed across multiple network units. Some or all units may be selected depending on the actual need to achieve the objectives of the solution of the embodiment.

Although the embodiments of the present disclosure have been described in detail above, the principles and embodiments of the present disclosure have been described herein using specific examples. It is only intended to support the understanding of its core concepts. At the same time, one of ordinary skill in the art may make changes in the embodiments and the scope of application of the present application by the concept of the present disclosure. Taken together, the content of the above specification should not be construed as limiting this disclosure.

Claims (23)

It is a convolution calculation method, and the method is used for a chip device.
The chip device receives input data and weights,
It said chip device includes the steps of the arrangement of the input data C are converted so that the innermost, obtain input data innermost is C,
The chip device includes a step of performing a convolution calculation on the input data whose innermost side is C and a weight, and obtaining the result of the convolution calculation.
Wherein the input data is four-dimensional data, the 4-dimensional N, H, W, is C, the dimension of the C is Ri dimensions der in the height direction of the four-dimensional data,
The chip device includes a master circuit and k slave circuits, and the chip device performs a convolution calculation on the input data and the weight whose innermost side is C, and the step of obtaining the result of the convolution calculation is Specifically,
The master circuit divides the weight into a plurality of basic data blocks, the master circuit distributes the plurality of basic data blocks to the k slave circuits, and the slave circuit stores the distributed basic data blocks. When,
The master circuit broadcasts each partial data of the input data to the k slave circuits, and the k slave circuits perform calculations on each partial data and the distributed basic data block, respectively, and obtain the calculation result. The step of obtaining and transmitting the calculation result to the master circuit,
The master circuit includes a step of obtaining the calculation result of the convolution operation according to all the operation results of the k slave circuits.
The k is the number of slave circuits, and the range of values of k is an integer of 2 or more.
A convolution calculation method characterized by that. The method according to claim 1.
Converts arrangement of the input data so C is innermost, the step of obtaining input data innermost is C, specifically,
Converts the arrangement of the input data so C is innermost, comprises obtaining input data NHWC or input data NWHC,
A convolution calculation method characterized by that. The method according to claim 1 or 2.
The weight is a four-dimensional data block, and the four dimensions of the weight are M, C, KH, and KW.
A convolution calculation method characterized by that. The method according to claim 3.
Specifically, when the M-dimensional value of the weight is m, the weight includes m convolution kernels, and the master circuit divides the weight into a plurality of basic data blocks.
The master circuit includes a step of dividing m convolution kernels into m basic data blocks.
A convolution calculation method characterized by that. The method according to claim 3.
Specifically, the master circuit divides the weight into a plurality of basic data blocks.
The master circuit comprises dividing m convolution kernels into m * c basic data blocks, where c is a value whose weight is in the dimension of C, and the range of values for c is an integer greater than or equal to 1. Is,
A convolution calculation method characterized by that. The method according to claim 4.
Specifically, the master circuit divides the weight into a plurality of basic data blocks.
When m> k, the master circuit divides m convolution kernels into m basic data blocks and distributes one or more of m basic data blocks to k slave circuits.
When m ≦ k, the master circuit includes a step of dividing m convolution kernels into m basic data blocks and delivering one of the m basic data blocks to k slave circuits.
A convolution calculation method characterized by that. The method according to claim 4 or 5.
The master circuit, broadcasting a respective partial data of the input data in the k-number of the slave circuit,
The master circuit cuts an input data block having the same size as the basic data block from the input data NHWC or input data NWHC as one partial data of each partial data, and inputs the input data block as a basic cut unit in step size. A step of cutting data to obtain each partial data and broadcasting one or more partial data of each partial data to k slave circuits is included.
A convolution calculation method characterized by that. The method according to claim 7.
The k slave circuits perform calculations on each partial data and the distributed basic data block to obtain calculation results, and the master circuit follows all the calculation results of the k slave circuits. To obtain the calculation result of the convolution operation, specifically
Each of the k slave circuits performs multiplication integration on the element value of one partial data and the element value of the corresponding position of the distributed basic data block, obtains a plurality of multiplication results, and obtains the plurality of multiplication results. The multiplication result of is transmitted to the master circuit, the master circuit accumulates the multiple multiplication results transmitted by each slave circuit to obtain multiple convolution results, and the master circuit sorts the multiple convolution results and calculates the convolution operation. Includes steps to get results , or
Each of the k slave circuits performs a multiplication operation on the element value of one partial data and the element value of the corresponding position of the distributed basic data block, obtains a plurality of multiplication results, and obtains a plurality of multiplication results. Including the step of accumulating the multiplication results of to obtain the convolution result, sending the convolution result to the master circuit, and the master circuit sorting all the convolution results to obtain the calculation result of the convolution operation.
A convolution calculation method characterized by that. The method according to any one of claims 1 to 6.
The chip device further comprises a branch circuit, the branch circuit connecting the master circuit to a plurality of slave circuits, and the method further comprises.
The branch circuit comprises transferring data between the master circuit and a plurality of slave circuits.
A convolution calculation method characterized by that. The method according to any one of claims 1 to 6.
The master circuit includes one or any combination of a vector arithmetic circuit, an arithmetic logic unit circuit, an accumulator circuit, a matrix transpose circuit, a direct memory access circuit, or a data sorting circuit.
A convolution calculation method characterized by that. The method according to any one of claims 1 to 6.
The slave circuit includes one or any combination such as an inner product arithmetic circuit or an accumulator circuit.
A convolution calculation method characterized by that. It ’s a chip device,
The chip device is used to receive input data and weights, the input data is four-dimensional data, the four dimensions are N, H, W, C , and the C dimension is the four dimensions. The height dimension of the data
The chip device is further used to convert the arrangement of input data so that C is on the innermost side to obtain input data on which C is on the innermost side.
It said chip unit perform convolutional arithmetic operations on the input data and the weighting innermost is C, which we used so as to obtain the calculation result of the convolution,
The chip device further includes a master circuit and k slave circuits.
The master circuit specifically divides the weight into a plurality of basic data blocks, distributes the plurality of basic data blocks to the k slave circuits, and broadcasts each partial data of the input data to the k slave circuits. Used as
The slave circuit stores a specifically distributed basic data block, performs an operation on each partial data and the distributed basic data block, obtains an operation result, and transmits the operation result to the master circuit. Used for
The master circuit is specifically used to obtain the calculation result of the convolution operation according to all the operation results of k slave circuits, where k is the number of slave circuits and the range of the value of k is 2 or more. Is an integer of
A chip device characterized by that. The chip device according to claim 12.
It said chip device, specifically, to convert the arrangement of the input data so C is innermost, used to obtain the input data NHWC or input data NWHC,
A chip device characterized by that. The chip device according to claim 13.
The weight is a four-dimensional data block, and the four dimensions of the weight are M, C, KH, and KW.
A chip device characterized by that. The chip device according to claim 12.
If the value of the weight in the M dimension is m, then the weight comprises m convolution kernels, and the master circuit specifically divides the m convolution kernels into m basic data blocks. Used for
A chip device characterized by that. The chip device according to claim 12.
Specifically, the master circuit is used to divide m convolution kernels into m * c basic data blocks, where c is a value in the dimension of C, and the range of values of c is 1 or more. Is an integer of
A chip device characterized by that. The chip device according to claim 15.
When m> k, the master circuit specifically divides m convolution kernels into m basic data blocks and distributes one or more of m basic data blocks to k slave circuits.
When m ≦ k, the master circuit specifically divides m convolution kernels into m basic data blocks and distributes one of the m basic data blocks to k slave circuits.
A chip device characterized by that. The chip device according to claim 15 or 16.
Specifically, the master circuit cuts an input data block having the same size as the basic data block from the input data NHWC or the input data NWHC as one partial data of each partial data, and uses the input data block as a basic cutting unit to perform steps. Cut the input data by size to obtain each partial data, and broadcast one or more partial data of each partial data to k slave circuits.
A chip device characterized by that. The chip device according to claim 18.
Specifically, the slave circuit performs a multiplication operation on the element value of one partial data and the element value of the corresponding position of the distributed basic data block, obtains a plurality of multiplication results, and obtains the plurality of multiplication results. Used to send the multiplication result to the master circuit
Specifically, the master circuit accumulates the plurality of multiplication results transmitted by each slave circuit to obtain a plurality of convolution results, and the master circuit sorts the plurality of convolution results to obtain a calculation result of the convolution operation. Used for
A chip device characterized by that. The chip device according to claim 18.
Specifically, the slave circuit performs a multiplication operation on the element value of one partial data and the element value of the corresponding position of the distributed basic data block, obtains a plurality of multiplication results, and obtains a plurality of multiplications. The results are accumulated to obtain the convolution result, the convolution result is sent to the master circuit, and the master circuit is used to sort all the convolution results and obtain the calculation result of the convolution operation.
A chip device characterized by that. The chip device according to any one of claims 13 to 17.
The chip device further includes a branch circuit, which connects the master circuit to a plurality of slave circuits.
The branch circuit is used to transfer data between the master circuit and a plurality of slave circuits.
A chip device characterized by that. The chip device according to any one of claims 13 to 17.
The master circuit includes one or any combination of a vector arithmetic circuit, an arithmetic logic unit circuit, an accumulator circuit, a matrix transpose circuit, a direct memory access circuit, or a data sorting circuit.
A chip device characterized by that. The chip device according to any one of claims 13 to 17.
The slave circuit includes one or any combination such as an inner product arithmetic circuit or an accumulator circuit.
A chip device characterized by that.

Images